Skip to content

Writing data from Node.js

Estimated time to read: 5 minutes

This page shows how to write Node.js application logs into EraSearch. In this guide, you'll:

  • Create a sample Node.js application with Winston as a logging framework.
  • Write data to EraSearch using the Elasticsearch transport.
  • View the logs in EraSearch.

Before you begin

This content is intended for engineers and developers using EraSearch on EraCloud or self-hosted EraSearch:

This page also assumes you've installed Node.js and jq, a JSON parser for the command line.

Instructions

Step 1: Create a sample Node.js application

  1. In your terminal, enter this command to initialize a node project in a new sample directory:

    $ mkdir sample && cd sample && npm init -y
    

    The command returns this output:

    Wrote to .../sample/package.json:
    
    {
      "name": "sample",
      "version": "1.0.0",
      "description": "",
      "main": "index.js",
      "scripts": {
        "test": "echo \"Error: no test specified\" && exit 1"
      },
      "keywords": [],
      "author": "",
      "license": "ISC"
    }
    
  2. Create a small server by adding this code to a new file named app.js:

    require("http")
      .createServer((_, res) => {
        res.end("Hello World");
      })
      .listen(3000);
    
  3. Start the server:

    $ node app.js
    
  4. To verify your setup, go to a separate terminal window and enter the command below. If the server is up and running, the command returns Hello World.

    $ curl localhost:3000
    

To stop the server, press Ctrl + C.

Step 2: Configure Winston

  1. In the sample directory, add Winston to your node project:

    $ npm install winston
    

  2. To create a Winston logger and emit your first log message, update app.js to look like the following:

    const logger = require("winston").createLogger();
    
    require("http")
      .createServer((_, res) => {
        logger.info("Message received");
        res.end("Hello World");
      })
      .listen(3000);
    

  3. To verify your configuration, restart the server and make the same $ curl localhost:3000 request.

    The server's log output shows that it received a document but doesn't have a logging backend. You'll set up the backend in step 3.

    [winston] Attempt to write logs with no transports {"message":"Message received","level":"info"}
    

Step 3: Configure an EraSearch transport

  1. In your sample directory, add the winston-elasticsearch transport to your node project:

    $ npm install winston-elasticsearch
    

  2. Configure the transport and add it to Winston.

    Add the following to the top of app.js, replacing YOUR_SERVICE_URI and YOUR_API_KEY with your EraCloud account information:

    const {
      ElasticsearchTransport,
      ElasticsearchTransformer,
    } = require("winston-elasticsearch")        
    const esTransport = new ElasticsearchTransport({
      transformer: (logData) => {
        const { ["@timestamp"]: _, ...transformed } = ElasticsearchTransformer(logData);
        return { ...transformed, _ts: Date.now() };
      },
      clientOpts: {
        node: "YOUR_SERVICE_URI",
        auth: {
          bearer: "YOUR_API_KEY",
        },
      },
    });
    

    Add the following to the top of app.js, replacing:

    • YOUR_ERASEARCH_URL with your EraSearch URL. Example: http://localhost:9200.

    • YOUR_API_KEY with your EraSearch RBAC API key. If you're not using RBAC, remove the auth section from the file.

    const {
      ElasticsearchTransport,
      ElasticsearchTransformer,
    } = require("winston-elasticsearch");
    
    const esTransport = new ElasticsearchTransport({
      transformer: (logData) => {
        const { ["@timestamp"]: _, ...transformed } = ElasticsearchTransformer(logData);
        return { ...transformed, _ts: Date.now() };
      },
      clientOpts: {
        node: "YOUR_ERASEARCH_URL",
        auth: {
          bearer: "YOUR_API_KEY",
        },
      },
    });
    
  3. Next, in app.js, update the existing logger declaration to tell Winston to use the new transport:

    const logger = require("winston").createLogger({
    transports: [esTransport],
    });
    

Note

The configuration above uses the Elasticsearch transport. That workflow is possible because EraSearch supports much of the Elasticsearch API. The example above replaces Elasticsearch's timestamp field (@timestamp) with EraSearch's timestamp field (_ts).

Step 4: View your data in EraSearch

  1. Restart the server:

    $ node app.js
    
  2. In a separate terminal window, send your server a request:

    $ curl localhost:3000
    
  3. View your data.

    Access EraSearch's UI by visiting your EraCloud account and clicking search icon. Your logs are in the index called logs-YYYY.MM.DD.

    Enter the command below in your terminal, replacing:

    • YOUR_ERASEARCH_URL with your information.
    • YYYY.MM.DD with today's date.
    • YOUR_API_KEY with your EraSearch RBAC API key. If you're not using RBAC, remove -H 'Authorization: Bearer YOUR_API_KEY' from the command.

    The request targets the logs-YYYY.MM.DD index, and returns one document with the field "message": "Message received".

    $ curl 'YOUR_ERASEARCH_URL/logs-YYYY.MM.DD/_search?q=_lid:*' \
    -H 'Authorization: Bearer YOUR_API_KEY' | jq
    

Next steps

You can extend your existing configuration to do the following:

  • Add custom fields to your log messages, for example, logger.info({message: "abc123", customField: "value1"}).
  • Configure a static index name by adding index: "myIndex" alongside transformer in your transport options.

For other ways to get data into your database, visit the write-integrations reference. To learn more about exploring, querying, and visualizing your data in EraSearch, visit these pages:


Last update: October 6, 2022