Julia : The performance booster

As we have entered into the time of cloud, nobody uses local storage now these days and I don’t think the next generation will ever know what a pen-derive is. Every second the data in the world is increasing in terms of TB and we have already started talking about data of size Petabyte(PB). Hence managing big amount of data is a minimum requirement for a programming language at this time.

Well in my 5 years of full stack development I have worked with PHP, JavaScript, Python,SQL and I think every language targets a particular problem and they have their own drawbacks as well. Like Python is very easy to learn, developers don’t have to worry about memory allocation or garbage collections. It is a general purpose programming language which can be used in scripting, automation, data analytics, visualizations, web development and so on. If you can understand English then you can understand Python. Yes, it is that easy to learn.

Print(“Python”) → simply prints “Python” in your output window.

But the problem is, because of dynamic memory allocation, it take a lot of space and time while processing bigger data. For example files with size in range of GB or higher.

Hence comes Julia in picture. Julia is designed to target the scientific computing domain. It is a fresh language creates in 2012 and caught developers eye from 2018. Julia is very similar to Python in terms of feature, structure which makes the learning curve short, and just behind “C” in the race of execution speed and performance which makes it unique. So far I have explored the language and read articles, I think SPEED and PERFORMANCE are the reasons why Julia came into existence even after we have numbers of stable language in the industry.

So lets see a demo.

I have 3 files with different sizes(21Mb, 547MB and 913MB) downloaded from kaggle and performed a simple operation of reading the CSV files and getting a filtered data. Below are the results :-

Code execution
Code execution

From the above execution we can say that :

  1. We should use Julia when we need to deal with a hug size of data. From the above result you can see for the big/bigger files their is a significant difference in the execution time. As it is meant for scientific computing if we use this language for a small size of data processing like 20 or 50 MB then it will not make any sense as you can see the processing time is nearly equal. Like we can break a piece of stick using our hands but to break a wall, we need a hammer. Even though hammer can break the stick we still use our hands.

So can we use Julia with my existing Python project?

YES. Julia can be integrated with Python by using pyjulia package. For example, lets say you have a web application where you need to generate a invoice which needs to do a lot of processing of excel files for formulas, csv files for getting values of different parameters, database interaction and finally some computations. You can build your invoice module in Julia and import this module into your Python (or Django views) so that processing time can be improvised. In this way you can kick start on Julia and slowly migrate towards it by implementing different modules one by one.

But lets see the the other side of the coin as well.

  1. Julia is 8 years old kid as compare to Python, who is 30 years old well established hunk. So its community is small. But like a kid its growing with time.
  2. JIT(Just in time): Because of which Julia program takes bit of time to load the packages you use for the first time of execution. But after that the performance speeds up. Its like the first gear problem when we learning driving. Once we manage to roll the vehicle from first gear, increasing speed becomes fun.

So take a cup of strong coffee and think through about the complexity of your project. If it is a simple one, then no need to make your life difficult by adding another dimension into it.

Finally the hot question … Will Julia replace Python? Well, I don’t know. Programming languages are just the tools we use to achieve our business so if the business demands to perform some operations which needs Julia then industry will adapt it in no time. For example, 10 years back we use to fill our details in websites and hit a SUBMIT button, but now we expect the web to read the data from the image we have. So as Sheldon says “the inevitability of change might be a universal constant”.

Thanks …….

Full stack Python Developer, Cloud engineer