Need of Automation Scripts?

What are Automation scripts ?

In our software world, we use different languages like Java, Python, Scala, Kotlin, .net, etc to develop a project. A project can be a web application, or a standalone application., or a mobile application., or a utility.

Depending on the functionalities required for the project, it might takes months to years. While developing the project, developers are very busy developing the code in high or low level programming languages. You might have noticed that sometimes a small piece of script can solve the problem instead of developing a sophisticated code with unit and integration tests and deploy to the server.

So what are these scripts? These scripts am mentioning about could be Shell/Bash script or even a Python script or other. Scripts can be 5 lines or 50 or even more…but yet powerful.

Sample script

Here is a simple script to read the files in a folder and print the file names.

for logfilename in "/opt/kafka/logs/*"
do
  echo "$logfilename"
done

If we would like to extend the same script, may be print the files with a pattern, there are several ways to do it for ex, find or grep. All you need to know is the right command., but it’s not so difficult to write one. Our google friend is always there.

One day there was a simple requirement at work, and started to write a python script, and that’s when I started to learn it. It slowly gained attention, and we team contributed to it, adding more functionalities, making it more reusable.

Do we really need these scripts?

Well, every now and then, looking at the other side of the Development, scripts are so efficient, easy to develop and deploy.

A Data engineer writes python scripts to crunch and transform the chaotic data into more readable datasets.

An Ops engineer writes shell scripts to build certain functionalities which can help in time and effort.

Tips for scripting

Go through Youtube/Udemy courses learning Unix/Linux commands in the beginning

Use http://regex101.com for regular expressions

Python tutorials : https://docs.python.org/3.7/tutorial/

Start with 5 lines of code and build up on that. That’s how you begin your journey. If you don’t have Mac or Unix environment, install a Virtualbox and play around.

In Kafka Context

Searching for a particular Acl subscription could be sometimes tedious job. Or adding a bunch of Acls for 100s of topics in a more automated way, scripts can be handy.

Finding topics with a particular retention period, or topics which are not used for a certain time period is very easy if you learn scripting.

Ansible scripts to install/deploy Kafka or any packages

Conclusion

As a Java developer, I like to write scripts sometimes which make life so easy. After-all scripts are so powerful if written and tested properly. Spend sometime now and then, exploring.

Have fun scripting.

Author : Murali Basani