# StandardSSLContextService configuration

**URL:** <https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96>\
**Category:** CogStack\
**Created:** [October 12, 2022, 6:42am UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96 "2022-10-12T06:42:02Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![rbeare](https://avatars.discourse-cdn.com/v4/letter/r/0ea827/32.png) [@rbeare](https://discourse.cogstack.org/u/rbeare)\
**Post date:** [October 12, 2022, 6:42am UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/1 "2022-10-12T06:42:02Z")

</div>

Hi,  
I’m attempting to run the workflows in the “ingest\_raw\_text\_into\_ES\_and\_annotate…” template. I’m stuck configuring the StandardSSLContextService configuration required by the PutElasticSearchHTTPRecord. Specifically, which trust and keystores should be used and what are the default passwords.

Thanks

---

<div class="post-metadata">

**Author:** ![vladd](https://avatars.discourse-cdn.com/v4/letter/v/8c91f0/32.png) [@vladd](https://discourse.cogstack.org/u/vladd)\
**Post date:** [October 12, 2022, 9:16am UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/2 "2022-10-12T09:16:28Z")

</div>

Hello,

Yep, the default is set to cogstackNifi, [Workflows — CogStack-Nifi latest documentation](https://cogstack-nifi.readthedocs.io/en/latest/deploy/workflows.html?highlight=cogstackNifi#apache-nifi-web-user-interface) .

The trust and keystore paths should be there by default, if somehow they aren’t there, the paths are:/opt/nifi/nifi-current/es\_certificates/opensearch/elasticsearch/elasticsearch-1/elasticsearch-1-keystore.jks and opt/nifi/nifi-current/es\_certificates/opensearch/elasticsearch/elasticsearch-1/elasticsearch-1-truststore.jks

If you are using elasticsearch native then change the “opensearch” bit of the path to “es\_native” .

---

<div class="post-metadata">

**Author:** ![rbeare](https://avatars.discourse-cdn.com/v4/letter/r/0ea827/32.png) [@rbeare](https://discourse.cogstack.org/u/rbeare)\
**Post date:** [October 12, 2022, 10:54am UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/3 "2022-10-12T10:54:21Z")

</div>

Thanks - that seems to work.

One problem I should document for anyone else searching is that even though the file suffix is jks, the “type” field needs to be set to PKCS12, which might be obvious to certificate experts.

---

<div class="post-metadata">

**Author:** ![rbeare](https://avatars.discourse-cdn.com/v4/letter/r/0ea827/32.png) [@rbeare](https://discourse.cogstack.org/u/rbeare)\
**Post date:** [October 12, 2022, 10:59am UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/4 "2022-10-12T10:59:43Z")

</div>

Spoke too soon.  
The test of the “basic ingestion DB to ES” gave the following error :

Any suggestions?

```auto
10:55:46 UTC

ERROR

f3aa6c49-015b-3007-a79a-813fca8d53ee

PutElasticsearchHttpRecord[id=f3aa6c49-015b-3007-a79a-813fca8d53ee] Routing to failure due to exception: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target - Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target - Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

```

---

<div class="post-metadata">

**Author:** ![vladd](https://avatars.discourse-cdn.com/v4/letter/v/8c91f0/32.png) [@vladd](https://discourse.cogstack.org/u/vladd)\
**Post date:** [October 12, 2022, 1:51pm UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/5 "2022-10-12T13:51:28Z")

</div>

Which version of elasticsearch are you using ?

I realised I made a mistake the original es\_native (the one that is currently the default) certs are as follows /opt/nifi/nifi-current/es\_certificates/es\_native/elasticsearch/elasticsearch-1/elasticsearch-1.p12 , use the same for both keystore and truststore, and as you mentioned set the type to PKCS12.

---

<div class="post-metadata">

**Author:** ![rbeare](https://avatars.discourse-cdn.com/v4/letter/r/0ea827/32.png) [@rbeare](https://discourse.cogstack.org/u/rbeare)\
**Post date:** [October 12, 2022, 9:18pm UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/6 "2022-10-12T21:18:33Z")

</div>

I’m using the default from the master branch, which the services.yml file says is 8.3.3

Ive set both to:

/opt/nifi/nifi-current/es\_certificates/opensearch/elasticsearch/elasticsearch-1/elasticsearch-1.p12

I’m getting the following error:

```auto
21:15:57 UTC
ERROR
f3aa6c49-015b-3007-a79a-813fca8d53ee

PutElasticsearchHttpRecord[id=f3aa6c49-015b-3007-a79a-813fca8d53ee] Routing to failure due to exception: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
- Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
- Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

```

---

<div class="post-metadata">

**Author:** ![vladd](https://avatars.discourse-cdn.com/v4/letter/v/8c91f0/32.png) [@vladd](https://discourse.cogstack.org/u/vladd)\
**Post date:** [October 12, 2022, 10:10pm UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/7 "2022-10-12T22:10:08Z")

</div>

I’ve edited the path in the previous answer, it is : /opt/nifi/nifi-current/es\_certificates/es\_native/elasticsearch/elasticsearch-1/elasticsearch-1.p12 after seeing that it was still pointing to the opensearch version.

Just for reference the certs are all mounted from the ./security/es\_certificates/ folder and respect the same convention on the paths.

---

<div class="post-metadata">

**Author:** ![rbeare](https://avatars.discourse-cdn.com/v4/letter/r/0ea827/32.png) [@rbeare](https://discourse.cogstack.org/u/rbeare)\
**Post date:** [October 12, 2022, 11:10pm UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/8 "2022-10-12T23:10:28Z")

</div>

I did try before (I noticed the same thing and was going to ask) and had a different error. I’ll test again later

---

<div class="post-metadata">

**Author:** ![rbeare](https://avatars.discourse-cdn.com/v4/letter/r/0ea827/32.png) [@rbeare](https://discourse.cogstack.org/u/rbeare)\
**Post date:** [October 13, 2022, 7:04am UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/9 "2022-10-13T07:04:16Z")

</div>

A different problem this time, although perhaps I can ignore it as it is a warning (see below). However I can’t seem to use the curl command to check elasticsearch.

```auto
06:46:24 UTC

WARNING

f3aa6c49-015b-3007-a79a-813fca8d53ee

PutElasticsearchHttpRecord[id=f3aa6c49-015b-3007-a79a-813fca8d53ee] Elasticsearch returned code 401 with message Unauthorized, transferring flow file to failure

```

The processor is set up to send to the default elasticsearch-1. However I can’t get any diagnostics out of it at all using stuff like. Do curl commands need to change now that there are certificates everywhere?

```auto
curl -s -XGET http://admin:admin@localhost:9200/medical_reports_text/_count

```

---

<div class="post-metadata">

**Author:** ![rbeare](https://avatars.discourse-cdn.com/v4/letter/r/0ea827/32.png) [@rbeare](https://discourse.cogstack.org/u/rbeare)\
**Post date:** [October 13, 2022, 8:23am UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/10 "2022-10-13T08:23:47Z")

</div>

Further - I managed to get a response via curl, but there are still problems. I’m guessing that the authentication is more complex now?

```bash
 curl -s -k -XGET https://admin:admin@localhost:9200/|jq

```

```json
{
  "error": {
    "root_cause": [
      {
        "type": "security_exception",
        "reason": "unable to authenticate user [admin] for REST request [/]",
        "header": {
          "WWW-Authenticate": [
            "Basic realm=\"security\" charset=\"UTF-8\"",
            "Bearer realm=\"security\"",
            "ApiKey"
          ]
        }
      }
    ],
    "type": "security_exception",
    "reason": "unable to authenticate user [admin] for REST request [/]",
    "header": {
      "WWW-Authenticate": [
        "Basic realm=\"security\" charset=\"UTF-8\"",
        "Bearer realm=\"security\"",
        "ApiKey"
      ]
    }
  },
  "status": 401
}

```

I note the following in services.yml:

```bash
 ELASTIC_USER=kibanaserver
 ELASTIC_PASSWORD=kibanaserver 

```

but can’t authenticate if I replace admin with kibanaserver.

---

<div class="post-metadata">

**Author:** ![vladd](https://avatars.discourse-cdn.com/v4/letter/v/8c91f0/32.png) [@vladd](https://discourse.cogstack.org/u/vladd)\
**Post date:** [October 13, 2022, 11:45am UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/11 "2022-10-13T11:45:36Z")

</div>

Although the services.yml contains the user/pass combo, the default superuser for elastic is “elastic” (this is unchangable), so the account that should work by default should be “elastic:kibanaserver” for the ES clusters (only the password is set by the default env var), to generate the rest of the users and the actual “kibanaserver” user you should use “/sercurity/create\_es\_native\_credentials.sh” (if you inspect the file you will see additional env vars that can be set for the user/pass combos) , I know this is only mentioned in the documentation as an optional step but it is now mandatory. I will be amanding the docs to reflect this.

If it still doesnt work I recommend deleting the containers and their respective volumes, start the containers and wait 30 secs before attempting to create the accounts.

---

<div class="post-metadata">

**Author:** ![rbeare](https://avatars.discourse-cdn.com/v4/letter/r/0ea827/32.png) [@rbeare](https://discourse.cogstack.org/u/rbeare)\
**Post date:** [October 13, 2022, 10:05pm UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/12 "2022-10-13T22:05:02Z")

</div>

I think we’re making progress here, but still work to do.

The current problem is with `create_es_node_cert.sh` (note that there are underscore in the name, missing from [docs](https://cogstack-nifi.readthedocs.io/en/latest/security.html).

There are problems adding entries to the keystore if the existing entries use the default alias (`mykey`) - see [here](https://www.ibm.com/support/pages/keytool-error-javalangexception-certificate-not-imported-alias-already-exists).

This causes the last command of `create_keystore.sh` to fail. I’m sure there’s a simple fix by deleting the appropriate files and starting from scratch.

```bash
echo "Creating truststore key"
keytool -import -file $1.pem -keystore $1-"truststore.key" -storepass $KEYSTORE_PASSWORD -noprompt

```

---

<div class="post-metadata">

**Author:** ![vladd](https://avatars.discourse-cdn.com/v4/letter/v/8c91f0/32.png) [@vladd](https://discourse.cogstack.org/u/vladd)\
**Post date:** [October 13, 2022, 10:38pm UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/13 "2022-10-13T22:38:02Z")

</div>

Since you are using ES native, you only need to execute create\_es\_native\_certs.sh, it will generate all the appropiate certificates .

From the security folder, the scripts that are for the ES native all have ‘es\_native’ in them, always look to these when you need something . We had to add these as a separate set of scripts because unfortunately the original ES needs its own tool for the certs.

---

<div class="post-metadata">

**Author:** ![rbeare](https://avatars.discourse-cdn.com/v4/letter/r/0ea827/32.png) [@rbeare](https://discourse.cogstack.org/u/rbeare)\
**Post date:** [October 13, 2022, 11:05pm UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/14 "2022-10-13T23:05:16Z")

</div>

Thanks - testing this now. I have questions about es native vs opensearch too, but I’ll post a separate topic.

No luck - now Kibana is causing problems:

```bash
 bash ./create_es_native_credentials.sh
ELASTIC_HOST not set, defaulting to ELASTIC_HOST=localhost
ELASTIC_PASSWORD not set, defaulting to ELASTIC_PASSWORD=kibanaserver
ELASTIC_USER not set, defaulting to ELASTIC_USER=elastic
KIBANA_USER not set, defaulting to KIBANA_USER=kibanaserver
KIBANA_PASSWORD not set, defaulting to KIBANA_PASSWORD=kibanaserver
INGEST_SERVICE_USER not set, defaulting to INGEST_SERVICE_USER=ingest_service
INGEST_SERVICE_PASSWORD not set, defaulting to INGEST_SERVICE_PASSWORD=ingest_service
Waiting for Elasticsearch availability
curl: (6) Could not resolve host: 
{
  "name" : "es01",
  "cluster_name" : "elasticsearch-cogstack-cluster",
  "cluster_uuid" : "nNx6zk33TOOipJTlPjB_ig",
  "version" : {
    "number" : "8.3.3",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "801fed82df74dbe537f89b71b098ccaff88d2c56",
    "build_date" : "2022-07-23T19:30:09.227964828Z",
    "build_snapshot" : false,
    "lucene_version" : "9.2.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

Setting kibana_system password
{"error":{"root_cause":[{"type":"unavailable_shards_exception","reason":"[.security-7][0] [1] shardIt, [0] active : Timeout waiting for [1m], request: indices:data/write/update"}],"type":"unavailable_shards_exception","reason":"[.security-7][0] [1] shardIt, [0] active : Timeout waiting for [1m], request: indices:data/write/update"},"status":503}Creating users
{
  "error" : {
    "root_cause" : [
      {
        "type" : "unavailable_shards_exception",
        "reason" : "[.security-7][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[.security-7][0]] containing [index {[.security][user-kibanaserver], source[{\"username\":\"kibanaserver\",\"password\":\"$2a$10$9ixjIyIdAYRo0Dlcq5tVLOJXhDtPrzcyd6jXZfygKL2OLAkqT9bGi\",\"roles\":[\"kibana_system\",\"kibana_admin\",\"ingest_admin\"],\"full_name\":\"kibanaserver\",\"email\":\"cogstack@admin.net\",\"metadata\":null,\"enabled\":true,\"type\":\"user\"}]}] and a refresh]"
      }
    ],
    "type" : "unavailable_shards_exception",
    "reason" : "[.security-7][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[.security-7][0]] containing [index {[.security][user-kibanaserver], source[{\"username\":\"kibanaserver\",\"password\":\"$2a$10$9ixjIyIdAYRo0Dlcq5tVLOJXhDtPrzcyd6jXZfygKL2OLAkqT9bGi\",\"roles\":[\"kibana_system\",\"kibana_admin\",\"ingest_admin\"],\"full_name\":\"kibanaserver\",\"email\":\"cogstack@admin.net\",\"metadata\":null,\"enabled\":true,\"type\":\"user\"}]}] and a refresh]"
  },
  "status" : 503
}

```

---

<div class="post-metadata">

**Author:** ![vladd](https://avatars.discourse-cdn.com/v4/letter/v/8c91f0/32.png) [@vladd](https://discourse.cogstack.org/u/vladd)\
**Post date:** [October 14, 2022, 8:59am UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/15 "2022-10-14T08:59:53Z")

</div>

Hmm, are both ES instances online ? You need to start both elasticsearch-1 and elasticsearch-2. The shard error is on the ES side, they both need to sync. Kibana should work fine afterwards and you should have no problems with generating the accounts.

---

<div class="post-metadata">

**Author:** ![rbeare](https://avatars.discourse-cdn.com/v4/letter/r/0ea827/32.png) [@rbeare](https://discourse.cogstack.org/u/rbeare)\
**Post date:** [October 14, 2022, 9:04am UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/16 "2022-10-14T09:04:49Z")

</div>

Docker ps does list both of them as alive. Is there any other way I should check?

```bash
af8f5dccdb60 docker.elastic.co/kibana/kibana:8.3.3 "/bin/tini -- /usr/l…" 10 hours ago Up 10 hours 0.0.0.0:5601->5601/tcp, :::5601->5601/tcp cogstack-kibana
8ada1aaa9b1d docker.elastic.co/elasticsearch/elasticsearch:8.3.3 "/bin/tini -- /usr/l…" 10 hours ago Up 10 hours 0.0.0.0:9201->9200/tcp, :::9201->9200/tcp, 0.0.0.0:9301->9300/tcp, :::9301->9300/tcp, 0.0.0.0:9601->9600/tcp, :::9601->9600/tcp elasticsearch-2
a6eaf2ece6b1 docker.elastic.co/elasticsearch/elasticsearch:8.3.3 "/bin/tini -- /usr/l…" 10 hours ago Up 10 hours 0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 0.0.0.0:9300->9300/tcp, :::9300->9300/tcp, 0.0.0.0:9600->9600/tcp, :::9600->9600/tcp elasticsearch-1
d086b162d0ea postgres:14.5-alpine "docker-entrypoint.s…" 3 days ago Up 10 hours 0.0.0.0:5554->5432/tcp, :::5554->5432/tcp cogstack-samples-db
bd898788ccaf cogstacksystems/tika-service:0.5.1 "/bin/bash /app/run.…" 3 days ago Up 10 hours 0.0.0.0:8090->8090/tcp, :::8090->8090/tcp cogstack-tika-service
e9f09170a565 deploy_nifi-nginx "/docker-entrypoint.…" 3 days ago Up 10 hours 80/tcp, 0.0.0.0:8443->8443/tcp, :::8443->8443/tcp cogstack-nifi-nginx
9225e080176c cogstacksystems/cogstack-nifi:latest "../scripts/start.sh" 3 days ago Up 10 hours 8000/tcp, 8080/tcp, 10000/tcp, 10443/tcp, 0.0.0.0:8082->8443/tcp, :::8082->8443/tcp cogstack-nifi

```

---

<div class="post-metadata">

**Author:** ![vladd](https://avatars.discourse-cdn.com/v4/letter/v/8c91f0/32.png) [@vladd](https://discourse.cogstack.org/u/vladd)\
**Post date:** [October 14, 2022, 3:19pm UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/17 "2022-10-14T15:19:53Z")

</div>

Nope, that was it, looks good! Could you check elasticsearch-1 or 2 logs ? if they seem fine and no access or connection issues are present then delete and re-create the kibana container. The shard errors only appear if somehow the setup had more than the two clusters active at some point (which was never the case here).

---

<div class="post-metadata">

**Author:** ![rbeare](https://avatars.discourse-cdn.com/v4/letter/r/0ea827/32.png) [@rbeare](https://discourse.cogstack.org/u/rbeare)\
**Post date:** [October 14, 2022, 9:39pm UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/18 "2022-10-14T21:39:50Z")

</div>

Any specific logs I should be checking. /var/log doesn’t appear to have anything specific to elasticsearch:

```bash
docker exec -it 8ada1aaa9b1d /bin/bash

elasticsearch@8ada1aaa9b1d:~$ ls /var/log/
alternatives.log apt bootstrap.log btmp dpkg.log faillog lastlog wtmp

```

The shell’s wd is `/usr/share/elasticsearch` and there’s a `log/gc.log`, and I can’t see any errors there. Anywhere else?

---

<div class="post-metadata">

**Author:** ![rbeare](https://avatars.discourse-cdn.com/v4/letter/r/0ea827/32.png) [@rbeare](https://discourse.cogstack.org/u/rbeare)\
**Post date:** [October 16, 2022, 10:54am UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/19 "2022-10-16T10:54:50Z")

</div>

I’m testing a fresh install using a new user with ID=1000, as a sanity check.

One thing I’ve noticed is that the test db has changed structure - no medical\_reports\_text anymore:

```auto
 psql -U test -d db_samples 
psql (14.5)
Type "help" for help.

db_samples=# \dt
             List of relations
 Schema | Name | Type | Owner 
--------+------------------+-------+-------
 public | annotations | table | test
 public | documents | table | test
 public | meta_annotations | table | test
 public | nlp_models | table | test

```

**Discovered the problem - the db dumps are managed by git lfs and aren’t necessarily retrieved by a default clone. a `git lfs pull` after the fact fixed this issue**

---

<div class="post-metadata">

**Author:** ![rbeare](https://avatars.discourse-cdn.com/v4/letter/r/0ea827/32.png) [@rbeare](https://discourse.cogstack.org/u/rbeare)\
**Post date:** [October 17, 2022, 9:59am UTC](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96/20 "2022-10-17T09:59:45Z")

</div>

Still struggling to get the elasticsearch dockers configured. What is the order of configuration scripts that I should be running and which ones should be run while the containers are running.

I just tried running `bash create_es_native_certs.sh` and then restarted the dockers but found that they are continually restarting, so something isn’t right.

[Next page](https://discourse.cogstack.org/t/standardsslcontextservice-configuration/96.md?page=2)