1. Install self-hosted EraSearch
Estimated time to read: 5 minutes
Acquisition notice
In October 2022, ServiceNow acquired Era Software. The documentation on this site is no longer maintained and is intended for existing Era Software users only.
To get the latest information about ServiceNow's observability solutions, visit their website and documentation.
Self-hosted EraSearch lets you run and manage EraSearch on your own cloud infrastructure. Use it to integrate EraSearch into your existing setup, lower costs, and get the most out of your log data.
This two-part series is intended for developer audiences, and it outlines how to install and use self-hosted EraSearch. In this series, you’ll:
- Set up and run EraSearch on your cloud infrastructure.
- Write data to EraSearch.
- Explore and query data using the EraSearch API.
Note
This series works with self-hosted EraSearch. If you're looking for the fastest way to set up and use EraSearch, get started with EraSearch on EraCloud.
Before you begin¶
Before you start, reach out to us at Era Software to get the following:
eradb-x.x.x.tgz
- Compressed files with Helm charts for EraSearch.values-eradb.yaml
- The Helm chart values file for EraSearch, customized for your deployment.eradb-registry-xxxx.json
- Authentication credentials for Era Software's container registry.
This guide assumes you're comfortable with Kubernetes, Helm, and EraSearch's high-level architecture. It also assumes you have the following in your environment:
- Kubernetes version 1.20+ with persistent volumes, and Helm.
-
Make sure they're installed on the machine you'll be deploying from.
-
An object storage provider.
Self-hosted EraSearch works with Amazon Web Services (AWS) S3, Google Cloud Storage, Microsoft Azure Blob Storage, or an API-compatible alternative.
Step 1: Set up the Kubernetes namespace¶
Follow these steps to create a Kubernetes namespace for the EraSearch deployment:
-
In your terminal, enter this command to create a new namespace, replacing
NAMESPACE_NAME
with your namespace name. -
Create a secret for storing Era Software's container registry credentials in the namespace. Enter this command, replacing:
NAMESPACE_NAME
with the namespace name you specified above.REGISTRY_SECRET
with the name of the JSON credential file you got from Era Software.REGISTRY_EMAIL
with the email address in that same file.
Step 2: Configure the Helm deployment¶
Make sure the values in values-eradb.yaml
are right for your EraSearch deployment.
For every database service, review these values:
-
image.repository
You may need to update the values if you're mirroring images from an internal registry instead of directly pulling them from Era Software's registry.
-
replicaCount
,resources.cpu
, andresources.memory
Make sure the values are valid for your Kubernetes cluster and deployment size.
Next, review these maxwell.treasurer
values, making sure they're valid for your environment and object storage budget:
-
monthly_budget
The maximum USD amount you want to spend on object storage per month.
-
batch_size
The maximum batch size, in bytes, after which EraSearch flushes data to object storage.
-
batch_delay_ms
The time, in milliseconds, after which EraSearch flushes data to object storage.
For AWS deployments¶
If you're using AWS, also review the following:
-
maxwell.s3.region
andmaxwell.s3.bucket
Set the values to the right bucket and region for your deployment.
-
maxwell.extraEnv
Make sure this section includes valid AWS credentials under the
AWS_ACCESS_KEY_ID
andAWS_SECRET_ACCESS_KEY
environment variables. For production deployments, Era Software recommends using Kubernetes service accounts with IAM pod roles instead of environment-provided credentials. -
quarry.persistence.storageClass
Set the value to the supported storage class. Era Software recommends starting with gp2. Visit AWS's Storage classes for more information.
For Microsoft Azure deployments¶
If you're using Microsoft Azure, also review the following:
-
maxwell.s3
Make sure the section specifies your Azure storage bucket, region, and endpoint (including the URI and SAS token). You may also want to check the network security settings for your storage account, ensuring EraSearch deployment requests can authenticate.
s3: bucket: your-container-name region: eastus2 endpoint: https://XXXX.blob.core.windows.net/?sv=2020-08-04&ss=b&srt=sco&sp=rwdlacitfx&se=2023-03-04T14:33:22Z&st=2022-03-04T06:33:22Z&spr=https&sig=XXXXXXXX
Note
While the name
s3
refers to Amazon's object storage service, the specification still works with Microsoft Azure Blob Storage. -
maxwell.extraEnv
In this section, set the following array elements:
-
quarry.persistence.storageClass
Set the value to supported storage class. Era Software recommends starting with
managed-premium
unless you need provisioned IOPS. Visit Microsoft Azure's Storage classes for more information.
Step 3: Install EraSearch¶
With the values set and validated, follow these steps to install EraSearch:
-
Enter the command below in your terminal, replacing:
NAME
with a name for the EraSearch database release (for example, era).X.X.X
with the version of the Helm chart you got from Era Software.NAMESPACE_NAME
with the Kubernetes namespace you created above.
-
The installation takes a few minutes to complete. Enter this command to check its status:
Once the installation completes, you'll see this message:
Step 4: Verify your installation¶
Follow these steps to verify your EraSearch installation:
-
Connect to the
quarry-deployment
service. Enter the command below to open a local port forward, replacing:NAMESPACE_NAME
with the Kubernetes namespace you used above.NAME
with the name of the EraSearch database release you used above.
-
Visit http://localhost:9200. When EraSearch is deployed and the API is ready for data, you'll see this JSON document with metadata about your instance:
You're all set. You've now installed self-hosted EraSearch in your environment.
Next steps¶
Next, visit 2. Write and query data to get the basics and use the EraSearch REST API. If you're ready to integrate EraSearch into your existing setup, visit the list of tools for writing real-time data.