Billing setup

This document outlines how billing is integrated with our billing providers (Stripe, AWS, Azure) and how the signup flows work.

How Pricing Works

We configure the prices for each of the available plans in the core.product_pricing table. All prices are stored per minute. Each cent in the pricing table is stored as 100k (100000) cents, since CrateDB does not support fixed decimal point numbers. i.e. $0.11 per minute turns into 1100000. The prices in this table are only used for displaying to the customer in CrateDB Cloud. What they are actually billed is configured in each of the billing provider’s marketplace systems.

When having to update prices, refer to the handy pricing calculator in SharePoint to avoid having to invent the wheel.

How consumption reporting works

There are two k8s CronJobs that generate and send consumption reports to our billing providers. The CronJobs call the API and all the logic of actually doing things lives in the cloud-api project.

The first job (consumption-tracking) iterates over all running clusters and adds entries into the core.consumption_tracking table every 15 minutes.

The 2nd job (consumption-report) iterates over all the entries in the core.consumption_tracking table and generates the actual reports that are sent to the correct billing provider. The logic is slightly different based on the provider, but in general we send the number of DTUs that you’ve used of your configured “plan”.

Note

DTUs stand for Database Transaction Units and were borrowed from Azure when the first billing integration was developed. Think of it as an abstract capacity unit where each DTU is configured to some CPU/RAM/Disk combo (i.e. 4 cores, 16GB Ram / 1 TB disk).

Stripe

Stripe is the newest and sanest of the billing providers that we have. Making changes in Stripe is generally easier than the others.

Warning

Stripe has a “test” mode, which is a sandbox and allows you to test things before prod releases. The test mode in Stripe is connected to our dev environment (i.e. all products / prices). You can control if you are in test or live mode using the button in the Stripe menu (on the left).

Signup process

Not much magic here. When a customer creates their first cluster, we create a customer entity in Stripe and ask for their CC details (which we also store in Stripe).

Then we create a Stripe subscription and add the correct plan/price based on the customers cluster plan. Usage is then submitted against this subscription and specific price. If a customer provisions multiple clusters in one subscription, we add new plans/prices to it in Stripe.

Changing prices

Login to the stripe dashboard and go to the Products section. You will see the usual dev/gp/so/io stuff in there. It is not possible to change existing prices in Stripe, you will always need to add a new price instead. To do so, look at how any of the existing prices are setup and just copy accordingly.

Once you’ve added the new prices, they will need to be updated in the core.product_pricing table in brain (the billing_dimension column). Hopefully it’s fairly self-explanatory how this works.

Strong Customer Authentication (SCA)

By default, Stripe handles the SCA. The user has to authenticate once on-session. This happens when the user provides their card details and we do the “test” payment. If for some reason the SCA doesn’t happen or the bank requests a reauthentication, an email is sent to the customer by Stripe. See the SCA spike for further information.

AWS

AWS Account

We use a separate AWS Account for the Marketplace listing: 155815986439 owned by team-product@crate.io. This account is a child account of the main crate AWS Account. You shoud login to the crate account and then switch roles to the Marketplace account (use role OrganizationAccountAccessRole).

Signup process

The customers sign up for CrateDB Cloud via AWS Using the AWS Marketplace (search for CrateDB). The signup process does two things:

  1. Redirects the customer to CrateDB Cloud. We then create a (pending) subscription and ask the customer to log in (if they haven’t yet).

  2. Sends us a webhook indicating that the subscription has been activated. This is async and can happen anywhere from a few seconds to several minutes later. Hence we keep the customer’s subscription as ‘pending’ until we get a confirmation.

Webhook setup

We receive the webhook about a subscription being activated from AWS SNS. The AWS SNS Configuration is in the crate-marketplace account (see above). This is setup manually (no terraform/salt).

Changing prices

Having switched roles to the crate-marketplace account, go to the AWS Marketplace management console and edit the CrateDB Cloud offer. The editing is self-explanatory, however keep in mind that it will submit a support request to AWS and they will verify your changes before it goes live (which can take a few business days).

Testing

When testing AWS offers, you cannot use the main crate AWS account for signup (this is in contract with Azure, where you can). Easiest is to use a personal AWS account that has billing setup - you must have a CC!

On prod, you can simply signup from the public-facing AWS offers, which is almost straightforward.

On dev, it is a bit more involved. As the CrateDB Cloud Development plan is not public, you must create a special offer for your own account to be able to use it. To do that:

  1. Note down your personal AWS Account id.

  2. Login to the AWS console using the crate account, and switch roles to the marketplace account.

  3. Go to the billing management console and then to the Offers tab.

  4. Create a custom offer for the account ID you wrote down in step 1, and the CrateDB Cloud Development plan.

  5. “Extend” the offer.

  6. Go to the offer and copy the link.

  7. Logout from the crate AWS account.

  8. Log back in to your personal account.

  9. Go to the link you copied.

  10. Complete the signup process.

Azure

The Azure integration is quirky in the sense that the plan is selected on the Marketplace itself. This means that you cannot go through a normal wizard to select what cluster you want to deploy, but that will have to be selected in Azure, and then passed on to us (this is saved in the plan column in the subscriptions table).

Marketplace Account

Georg Traar and Romanas Sonkinas have the keys to the Marketplace Account where everything is configured. Changing prices happens in the azure partner portal.

Users and permissions are also managed separately in the user management page.

Signup process

The user selects the plan they want in the Azure marketplace. They are then redirected to CrateDB Cloud to /api/v2/azure/signup/ with a bunch of parameters that we validate. A new subscription is then created with the plan pre-selected.

Changing prices

Prices for published plans cannot be changed, so every time you change prices you need to create new plans. This is no big deal, as you can re-use old billing dimensions. Once the plans are created, the plan ids needs to be updated in the core.product_pricing table.