TheHive and Cortex on AWS — AMI tutorials part 3a: Launching Cortex manually with the AWS console

This is the third article in a series focusing on deploying TheHive and Cortex AMIs on AWS. We are publishing these tutorials alongside our AMI user guides to better illustrate what real-life deployments look like.

In part 1, we covered the reference SecOps VPC architecture used as the target environment in which we will launch our TheHive and Cortex instances.

In part 2, we used Terraform code to automatically launch TheHive and Cortex instances in our VPC.

In part 3, we will deploy the exact same thing as we did in part 2 but we will do so manually using only the AWS console. If you are not [yet] using Terraform, this tutorial is for you! This part is split in two (part 3a for Cortex and 3b for TheHive) to keep a reasonable article length.

We will use the same SecOps VPC we set up in part 1 as our target environment. You will need an existing VPC to follow along. It can differ from our reference architecture but you should at least already be running a public-facing system that will enable https for our instances.

The required security groups should also be created beforehand. The rules for each component are shown in the diagram below and fully documented in our AMI user

Since we will expect TheHive to connect to Cortex, we will start by launching the Cortex instance. We will be deploying both instances with empty databases.

Once our instances are deployed, this is what our VPC will look like:

Before you can launch a Cortex instance for the first time, you must subscribe to the AMI. Please visit the Cortex product page on the AWS Marketplace to proceed with the subscription and make the AMI available for use in your account.

• On the AMI product page, click on Continue to subscribe

• Click on Accept Terms

Wait a few minutes for the subscription to be effective (it’s usually very quick).

• Click on Continue to Configuration once it is enabled

• Click on Continue to Launch

• Under Choose Action, select Launch through EC2 and then click Launch

At this point, you will be taken to the EC2 console just as if you had selected the AMI from the public AMI list. You can easily find the Cortex AMI again in the future by applying a search filter on Cortex3.

• Select your instance type and click Next: Configure Instance Details

• Select your VPC under Network and private subnet under Subnet

• Scroll down to User data and input the following bootstrap script:

#cloud-config

The bootstrap script will partition and format both new EBS volumes (the Cortex database volume and the Docker volume) and launch the Cortex initialization script with both volumes as arguments. This script is fully documented in the Cortex AMI user guide.

• Click on Next: Add Storage

The Cortex AMI is designed to use three EBS volumes:

• The root filesystem volume
• The Cortex database volume (/dev/sdh — 30 GB by default)
• The Cortex Docker volume (/dev/sdi — 20 GB by default)

While the root filesystem is designed to be disposable, just like a container, both the Cortex database and Docker volumes are set to be persistent upon instance termination. Again, please refer to the Cortex AMI user guide for more information on these EBS volumes.

We recommend you enable disk encryption on both the Cortex database and Docker volumes. You can use the default KMS key if you are not already using custom keys.

• Click on Next: Add Tags

Add tags as needed (optional).

• Click on Next: Configure Security Groups

Here we need to select one or more security groups to allow the required network traffic to reach our Cortex instance. If you are using our SecOps VPC, the security groups to select are secops-cortex-sg and secops-ssh-sg. Again, these security groups are fully detailed in the Cortex AMI user guide but basically we need to allow:

→ incoming http traffic on port 9001 from the public-facing system (the load balancer in our case) and from our TheHive instance
→ SSH access on port 22 from our bastion host

• Click on Review and Launch

• Check if everything looks good and click on Launch

• Select your keypair and click on Launch Instances

Our Cortex instance is now up and running but we cannot yet connect to it since it is running in a private subnet. You must now route incoming traffic to this instance using your existing load balancer and / or reverse proxy.

AMI user guide

Wait a few seconds for the instance to launch and access the Target Group section of the EC2 console.

• Click on the secops-alb-cortex-tg target group

• Click on Register targets

• Select your Cortex instance, click on Include as pending below and then click on Register pending targets

• The target is registered with an Initial status

• Once the target is successfully registered and has passed the health check, the status will change to healthy

If you are using our SecOps VPC security groups, make sure your local IP address is whitelisted in the secops-lb-access-sg source IPs (or allow 0.0.0.0/0 as source if you wish to have your instance reachable from the whole internet without IP filtering).

That’s it; your Cortex instance should now be reachable and available for initialization. Before doing so, we still have one last action to perform in the console: registering a private DNS record for the Cortex instance.

Over time, you will probably replace your Cortex instance several times (at least if you follow our recommended upgrade procedure!). Every time you will launch a new instance, its ip address in the private subnet will change, which is why we will register a private DNS record so we can easily resolve the instance’s ip address without having to reconfigure stuff every time it changes (such as our TheHive integration with Cortex or our SSH config file). You could of course also manually manage static private ip addresses but we won’t cover that tortured path.

• Go to the Route53 console and access the private DNS zone associated with your VPC. Create a new A record for your Cortex instance such as cortex.secops.cloud with the private ip address.

If you have set up a bastion host configuration similarly to our reference architecture, you can now seamlessly SSH into your Cortex instance using the proxyjump functionality of the ssh client. The easiest way to do that is to create (or update) the ~/.ssh/config file. Use the example below as a reference and replace the bastion ip address, Cortex hostname and private key information.

You will now be able to SSH into the Cortex instance directly using the bastion host as a proxy:

ssh cortex
Note: Remember to whitelist your local public IP address in the bastion security group secops-public-ssh-sg.

Your Cortex instance is now awaiting behind your https listener, such as https://[secops_r53_cortex_record variable value] if you are using our SecOps VPC.

As we launched a brand new Cortex instance with an empty database, we now need to create:

• a Cortex administrator,
• a Cortex organization and
• an organization user (with an API key to connect from TheHive)

Follow the Cortex Quick Start guide for detailed instructions on how to do this. Start at step 2 (Update the Database) since Cortex is already set up and running.

Congratulations, you are done setting up Cortex!

We can now do pretty much the same with TheHive in part 3b. We will simply have one more step to configure the Cortex integration in the TheHive application configuration file. To do so, you will need an existing API key from an organization in your Cortex setup. You should create one at this point and store the API key for later use. For instance, you can create a user named thehive and give it the read and analyze roles.