Conveyor offers plans adapted to each type of user.
Plans range from Free, enabling you to start quickly, all the way to Enterprise, allowing you to meet strict compliance regulations.
AWS and Azure
The pricing is for the Conveyor platform only. It does not include pricing for any required cloud resources (e.g. compute instances, databases, ...). The baseline cloud cost for a single cluster is about 10 USD per day or about 300 USD per month.
Use the following steps to estimate the Conveyor and instance costs for a particular workload.
Jobs and environments
Let’s say you have two environments (staging and production). Each environment runs 10 DAGs with each one an average of 20 tasks. We will calculate the monthly costs for only the production environment were the jobs run daily.
In our example, the 20 tasks are split into 2 types:
10 Spark jobs running one mx.small driver and five mx.medium executors for 30 min.
10 container jobs running one mx.medium instance for 10 min.
Compute the number of mx.medium hours
Conveyor proposes multiple job compute types. In order to simplify calculations, we convert any types's number of hours of compute back to equivalent mx.medium hours. By definition, this type has 1 cpu core and 4 GiB mem.
The mx.small has 0.5 cpu cores, so this means that 1 hour of this instance is equivalent to 0.5 hours of an mx.medium.
Let’s do it for the production environment.
Spark executor: 10 DAGs * 10 tasks * 5 executors * 1/2hour* 30days = 7500 mx.medium hours
Spark driver: 10 * 10 * 1/2 hours *30 = 1500 mx.small hours = 750 mx.medium hours.
Container jobs: 10 * 10 * 1/2 hours *30 = 1500 mx.medium hours
Monthly you will use 9750 mx.medium hours.
Conveyor license costs
The Conveyor license costs are computed in core-hours. Because the mx.medium has a single cpu core, you can reuse the total amount of mx.medium hours without any additional multiplication.
For the production environment this means that 9750 instance hours will cost you 487.5 EUR in licensing cost for one month using the Conveyor Team plan.
An mx.medium job is a Conveyor concept. The underlying Kubernetes cluster itself can run on many types of instances. Kubernetes will scale the number of actual worker nodes for you based on the required number of cpu cores and memory. This means that for an estimation, you will need to roughly convert the mx.medium (1 cpu, 4 GiB mem) to the actually provisioned cloud instance.
Let’s say on AWS a mx.medium job is running on a m6i.mxlarge (4 cpu, 16 GiB mem) instance with an hourly cost of 0.214 EUR. A single mx.medium job hour is then roughly equivalent to 0.25 of a m6i.mxlarge hour, thus costing 0.0535 EUR per hour.
For our production environment, this means that the 9750 instance hours results in a cost of 521.63 EUR when running on-demand.
If you would run on spot instances, we can reduce this cost up to 90%, but let’s take a more conservative 70% reduction: 156.48 EUR.
This principle is the same on any cloud provider.
The total conveyor cost is the sum of the license and the cloud cost.
Our monthly total cost for the production environment would be 1009.13 EUR when running on-demand, and 643.98 EUR when running on spot instances.