Resolving Gremlin Console 3.4.4 and 3.3.9 error when connecting to CosmosDB Graph API

While following the quick-start describing how to connect to Cosmos DB graph database from Apache TinkerPop Gremlin console, I encountered the following error when using the latest stable release 3.4.4 and the latest maintenance release 3.3.9 (as of January 2, 2020).

Problem

Gremlin Console 3.4.4 to CosmosDB Error

When starting Gremlin console on Windows using the default bin\gremlin.bat , “Display stack trace? [y/N]” does not work due to a long open issue GROOVY-6453.

To be able to see the stack trace, I following the suggestion provided here (i.e. removing the -Djline.terminal=none option from gremlin.bat).

The stack trace was as follows:

Display stack trace? [yN]y
org.apache.tinkerpop.gremlin.jsr223.console.RemoteException: Host did not respond in a timely fashion - check the server status and submit again.
        at org.apache.tinkerpop.gremlin.console.jsr223.DriverRemoteAcceptor.submit(DriverRemoteAcceptor.java:188)
        at org.apache.tinkerpop.gremlin.console.GremlinGroovysh.execute(GremlinGroovysh.groovy:110)
        at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:120)
        at org.codehaus.groovy.tools.shell.Shell$leftShift$1.call(Unknown Source)
        at org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:93)
        at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:101)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1217)
        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:144)
        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:164)
        at org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:138)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:43)
        at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:190)
        at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:58)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:160)
        at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:57)
        at org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:101)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1217)
        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:144)
        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:164)
        at org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:97)
        at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:234)
        at org.apache.tinkerpop.gremlin.console.Console.<init>(Console.groovy:168)
        at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:234)
        at org.apache.tinkerpop.gremlin.console.Console.main(Console.groovy:502)
gremlin>

My Gremlin console configuration (conf/remote-secure.yaml) looked as follows:

hosts: [avgremlin1.gremlin.cosmos.azure.com]
port: 443
username: /dbs/{my_database}/colls/{my_collection}
password: MY_PRIMARY_COSMOSDB_ACCOUNT_KEY_ISHERE
connectionPool: { enableSsl: true }
serializer: { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV2d0, config: { serializeResultToString: true }}

Looking for Root Cause

Since I didn’t know which version of Gremlin console stopped working properly, I tried a few older versions in a “binary search” fashion:

• Try #1: 3.4.4 (latest stable release from October 2019) => fails
• Try #2: 3.3.4 (old version that should work) => works
• Try #3: 3.4.0 => works
• Try #4: 3.4.2 => works
• Try #5: 3.4.3 => works
• Try #6: 3.3.9 (latest maintenance release from October 2019) => fails

Since both of the most recent releases were failing to connect to Cosmos DB graph Gremlin endpoint, I looked at the 3.4.4 change log to see what could be causing this.

Among multiple improvements and fixes, the change log mentioned bug “fix” TINKERPOP-2289 (Use address instead of hostname for connection)…which most likely caused the problem.

Based on the JIRA and mailing list discussions, Apache TinkerPop Pull Request #1213 is reverting the problematic change in future releases 3.4.5 and 3.3.10.

Solution

Until versions 3.4.5 and 3.3.10 of Apache TinkerPop Gremlin Console are released, we can use the previous versions 3.4.3 and 3.3.8 to workaround the bug in the current version 3.4.4.

Thank you!

Please leave feedback and questions below or on Twitter https://twitter.com/ArsenVlad