Skip to content

Writing data with Cloudflare

Estimated time to read: 4 minutes

Acquisition notice

In October 2022, ServiceNow acquired Era Software. The documentation on this site is no longer maintained and is intended for existing Era Software users only.

To get the latest information about ServiceNow's observability solutions, visit their website and documentation.

Important

This integration is not yet generally available. Please reach out if you would like to use this integration or have any questions.

This page shows how to use Cloudflare's Logpush feature to write logs to EraSearch. In this guide, you'll:

  • Configure Cloudflare to push logs to EraSearch using the Splunk HEC format
  • View the logs in EraSearch

Before you begin

This content is intended for engineers and developers using EraSearch on EraCloud or self-hosted EraSearch:

You also need administrative access to a Cloudflare Enterprise account to configure the Logpush integration.

Instructions

Step 1: Go to the Logpush job configuration page in Cloudflare

From the Cloudflare dashboard, after selecting a domain you want to use with Logpush:

  1. Navigate to the Analytics section
  2. From Analytics, choose the Logs sub-section
  3. From Logs, select Connect a Service to create the integration

Step 2: Configure the data fields for your Logpush job

From the Select Data Set screen, choose the HTTP requests option.

From the Select Data Fields screen, select all available fields using the checkbox at the top.

Note

If you already know what data fields you want to include or exclude from the integration, feel free to only select those items.

If you are new to this integration, we recommend starting with all options.

Step 3: Configure the destination for your Logpush job

From the Select Destination screen, choose the Splunk option.

Note

This guide uses the Splunk option to integrate with EraSearch. That workflow is possible because the EraSearch REST API supports ingesting data in Splunk's HEC format.

From the Enter Destination Info screen, fill in the following details:

  • Splunk raw HTTP Event Collector URL - This is your EraSearch URL with the suffix /services/collector/raw. For example, if your EraSearch URL is https://era.example.com, set this option to https://era.example.com/services/collector/raw.

  • Channel ID - A random UUID or string identifier used to identify this stream of data. You'll need to generate this yourself using a random string or UUID generator.

  • Auth Token - This is the token set in the Authorization header of the incoming requests. To have EraSearch accept the data, this needs to be set to a valid authorization header.

    • For EraSearch on EraCloud, use Bearer%20${YOUR_ERACLOUD_TOKEN}, where ${YOUR_ERACLOUD_TOKEN} is your EraCloud API key.
    • For self-hosted EraSearch, insert a valid basic HTTP authorization header.

  • Source Type - The source type, set to cloudflare:json.

Step 4: Enable your Logpush job

With the destination information configured, all that's left to do is enable the integration by selecting Push.

Once completed, data should now be flowing from Cloudflare into your EraSearch database. In the next section, you'll view your data.

Step 5: View your data in EraSearch

Access EraSearch's UI by visiting your EraCloud account and clicking search icon. Your logs are in the logs-cloudflare index. You may need to refresh the UI if the index is new.

Use the EraSearch REST API to query the logs in EraSearch. Paste this command in your terminal, replacing YOUR_ERASEARCH_URL with your EraSearch URL, for example, http://localhost:9200.

$ curl 'YOUR_ERASEARCH_URL/logs-cloudflare/_search?q=_lid:*'

The response shows information about your data and API request, including:

  • took - The time, in milliseconds, EraSearch took to serve the query request
  • _id - A unique, auto-generated numerical identifier for documents

Next steps

You're all set. Your EraSearch instance is now receiving real-time log data. For more information about Cloudflare, including what logs you can collect, visit these pages:

For other ways to get data into your database, visit the write-integrations reference. To learn more about exploring, querying, and visualizing your data in EraSearch, visit these pages:

Last update: August 7, 2023