adam bien's blog

Monitoring Payara / Glassfish with Prometheus 📎

Firehose v0.0.3 supports "extractor" scripts (Nashorn's implementation of Java's java.util.function.Function interface) which can be used for extraction of arbitrary monitoring sources. e.g. to convert GlassFish's / Payara's the "Average request processing time" metric: curl http://localhost:4848/monitoring/domain/server/http-service/server/request/processingtime.json available under JSON:

{"message":"","command":"Monitoring Data","exit_code":"SUCCESS","extraProperties": {"entity":{"processingtime":{"unit":"millisecond","lastsampletime":1511693682663,"name":"ProcessingTime","count":2, "description":"Average request processing time","starttime":1511693222100}},"childResources":{}}}

The following script extracts the value of the "count" key, and provides additional metadata for prometheus.io metric (avgRequestProcessingTime.js):


function apply(input) {
    var metric = JSON.parse(input);
    var value = metric.extraProperties.entity.processingtime.count;
    var output = {
        suffix: "avg",
        units: "ms",
        component: "requestprocessingtime",
        application: "problematic",
        value: value
    };
    return JSON.stringify(output);
}

A payara with installed firehose (28 kB Thin WAR) ships as docker image from docklands.

The image can be started with: docker run -d -p 4848:4848 -p 8090:8090 --name payara-firehose airhacks/payara-firehose. The port 8090 is opened in addition to 8080 for monitoring purposes only.

The following PUT-request sets the monitoring targets and uploads the extractor script at the same time:

curl -i -XPUT -H"Content-type: text/plain" -H"uri: http://localhost:4848/monitoring/domain/server/http-service/server/request/processingtime.json" --upload-file avgRequestProcessingTime.js http://localhost:8090/firehose/resources/configurations/avgProcessingTime

Now: curl http://localhost:8090/firehose/resources/metrics should respond with something like:

problematic_requestprocessingtime_ms_avg 30

firehose acts as a gateway, so you only have to setup a single target at prometheus side:


(...)
scrape_configs:
  - job_name: 'firehose'
    scrape_interval: 5s
    metrics_path: '/firehose/resources/metrics'
    static_configs:
      - targets: ['payara-firehose:8090']

See you at Java EE Microservices and Java EE Troubleshooting, Performance and Monitoring

Is Munich's airport too far? Learn from home: javaeemicro.services