JobRunr, a relatively new JVM job scheduling tool, allows developers to schedule a job using just a one-line Java lambda that can manage delayed and recurring jobs.
JobRunr stores the job details for each job using a StorageProvider interface and supports all major SQL databases and NoSQL databases. In this way, it is fault-tolerant since, by default, it will reschedule the background job with an exponential back-off policy if it encounters an exception. Since JobRunr does not keep any locks on other jobs, it is possible to distribute and process the jobs on different machines.
This past week, JobRunr has released version 4.0 which brings updated Spring Boot Starter support, new integrations with Quarkus and Micronaut and a new jobs analysis performance mode that checks if a job can be cached to speed up the subsequent calls.
JubRunr also includes a built-in dashboard that allows monitoring of jobs.
Ronald Dehuysser, founder and main developer of JobRunr, spoke to InfoQ about this latest release
InfoQ: What was the inspiration for the development of JobRunr?
Ronald Dehuysser: On a couple of occasions in my professional life, I needed to make some really difficult calculations on several gigabytes of data that take some time to finish. And for these types of calculations, I never found the right Java framework to assist me in getting the job done reliably and in a clean code manner.
Another use case example is data import from legacy systems that do not support push notifications. But if you work in a load-balanced environment, this becomes a lot more difficult as all nodes will check for new data whereas this should only happen on one node.
There is Spring Batch or Quartz, but these frameworks all make you implement custom interfaces and they add a lot of overhead, whereas I just want to run some long-running tasks in the background.
JobRunr solves all of this by just accepting any Java 8 lambda, analyzing it and storing the job information in a SQL or NoSQL database. You can schedule these jobs to be executed as soon as possible, somewhere in the future or in a recurring manner using Cron expressions.
As soon as these jobs are saved in the database, they can then be processed in parallel on different servers. This decreases the total amount of time needed to process all these jobs. As it internally uses optimistic locking, each job will only be processed once.
In any software architecture, things can sometimes go wrong and will go wrong. Also, in this case, JobRunr helps and retries the failed jobs automatically with a smart backoff policy.
Last but not least, I wanted visibility in how these jobs were executed. That's why I added the dashboard. It really helps to monitor job progress and you can easily find the root cause of why a job failed.
InfoQ: What's on the horizon for JobRunr?
Dehuysser: JobRunr 4.0.0 was just released which adds better integration with Spring, Quarkus, and Micronaut. It also adds a new cache for the job analysis which makes things even faster.
As JobRunr is an open-source project. I'm open to ideas from the community on how to improve it. Some ideas that I have include a better algorithm (perhaps based on multivariate machine learning) to fasten the processing time of all these jobs.
InfoQ: What else would you like our readers to know about JobRunr?
Dehuysser: I would like to highlight three things:
- JobRunr does some magic with ASM (which is also used by Spring, Hibernate and a lot of other frameworks) to analyze the job lambda. By using ASM, I really learned a lot about the JVM bytecode, which is not as difficult as I imagined.
- As JobRunr performs bytecode analysis, it also participates in the Oracle Quality Outreach program. This means JobRunr is tested against upcoming releases of the JVM. This helps me to make sure that it will continue working on newer Java releases and also helps the Java community as bugs in the JVM itself are caught earlier.
- For users who need support or extra features, there is also JobRunr Pro. It builds on top of JobRunr and adds extra features like queues with different priorities (high priority jobs get processed before low priority jobs), job chaining, atomic batches and a better dashboard which adds search capabilities.
And, for any license sold, some new trees are planned as 5% of the revenue goes to teamtrees.
JobRunr may be compared with alternative tools like Quartz and Spring (Scheduling support). It is open-source and free for commercial use under the LGPLv3 license. A PRO version is also available, which adds features such as priority queues and job chaining.