Jetty web server customisation: common use cases

Managed by | Updated 5 years ago.

Prerequisites 

The following article assumes that a working Funnelback 15 server is available. The term $SEARCH_HOME is used throughout the document and this usually refers to either /opt/funnelback (Linux) or <drive>:\funnelback (Windows).

Before starting check your Funnelback version and the corresponding documentation to ensure that the CustomiseJettyServers.groovy script is available.

Objective

To provide a worked example of how to set up customJettyServers.groovy to deal with some common embeddedJetty customisations, i.e. host custom keystore/truststore, excluding protocols and including Cipher Suites.

Steps

• Login into the server that Funnelback is installed.
• Create $SEARCH_HOME/web/conf/customiseJettyServers.groovy in a text editor.
• Enter the following into the file:

import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.server.ServerConnector;
import org.eclipse.jetty.util.ssl.SslContextFactory;
import org.eclipse.jetty.server.HttpConfiguration;
import org.eclipse.jetty.server.HttpConnectionFactory;
import org.eclipse.jetty.server.Connector;
import org.eclipse.jetty.util.log.Log;

def removeConnectors(Map<String, Server> servers, String serverName) {

    //Finding curently set connectors for the  server context
    Connector[] publicConnectors = servers.get(serverName).getConnectors();


    //Removing ALL set connectors - to be replaced by the ones we create later!
    publicConnectors.each {
        servers.get("public").removeConnector(it);
    }
}

def HttpConnectionFactory createHttpConnectionFactory() {
    HttpConfiguration httpConfig = new HttpConfiguration();
    httpConfig.setOutputBufferSize(32768);
    httpConfig.setRequestHeaderSize(8192);
    httpConfig.setResponseHeaderSize(8192);
    HttpConnectionFactory httpConnectionFactory = new HttpConnectionFactory(httpConfig);
    return httpConnectionFactory;
}


def SslContextFactory createSslContextFactory(String keyStorePath, String keyStorePassword, String trustStorePath,String trustStorePassword,String[] excludeProtocols,String[] includeCipherSuites) {

    SslContextFactory sslContextFactory = new SslContextFactory();
    sslContextFactory.setKeyStorePath(keyStorePath);
    sslContextFactory.setKeyStorePassword(keyStorePassword);
    sslContextFactory.setTrustStorePath(trustStorePath);
    sslContextFactory.setTrustStorePassword(trustStorePassword);

    sslContextFactory.addExcludeProtocols(excludeProtocols);
    sslContextFactory.setIncludeCipherSuites(includeCipherSuites);

    return sslContextFactory;
} 


def Map<String, Server> customise(Map<String, Server> servers) {
    //Removing existing connectors for the public server context
    removeConnectors(servers, "public");

    // Add a HTTP connector to the public server
    ServerConnector httpConnector = new ServerConnector(servers.get("public"), createHttpConnectionFactory());
    httpConnector.setPort(9080);

    String[] excludeProtocols = ["SSL","SSLv2","SSLv2Hello","SSLv3"];
    String[] includeCipherSuites = ["TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384","TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256","TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384","TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256","TLS_DHE_RSA_WITH_AES_256_GCM_SHA384","TLS_DHE_RSA_WITH_AES_128_GCM_SHA256","TLS_DHE_RSA_WITH_AES_256_CBC_SHA256","TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA","TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA","TLS_DHE_RSA_WITH_AES_256_CBC_SHA","TLS_RSA_WITH_AES_256_GCM_SHA384","TLS_RSA_WITH_AES_128_GCM_SHA256","TLS_RSA_WITH_AES_256_CBC_SHA256","TLS_RSA_WITH_AES_128_CBC_SHA256","TLS_RSA_WITH_AES_256_CBC_SHA"];

    //SslContextFactory contstruction - to be shared by both admin and public; create a separate one if the properties used are different
    SslContextFactory sslContextFactory = createSslContextFactory(
              <$SEARCH_HOME>/web/conf/keystore","OBF:<keystore password>",
              <$SEARCH_HOME>/web/conf/keystore (** truststore **)","OBF:<trustore password>",
              excludeProtocols,includeCipherSuites);

    //An an SSL Connector to the public server context
    ServerConnector publicSslConnector = new ServerConnector(servers.get("public"), sslContextFactory);
    publicSslConnector.setPort(9843);

    //Adding http and https connectors to the public context
    servers.get("public").addConnector(httpConnector);
    servers.get("public").addConnector(publicSslConnector);

    Log.getRootLogger().info("CustomiseJettyServers - public server info:");
    Log.getRootLogger().info("Public SSL Exclude Cipher Suites:"+sslContextFactory.getExcludeCipherSuites());
    Log.getRootLogger().info("Public SSL Include Cipher Suites:"+sslContextFactory.getIncludeCipherSuites());
    Log.getRootLogger().info("Public Port:"+publicSslConnector.getPort());

    //Finding currently set connectors for the admin server context
    removeConnectors(servers,"admin");  

    //Add an SSL Connector to the admin server context
    ServerConnector adminSslConnector = new ServerConnector(servers.get("admin"), sslContextFactory);
    adminSslConnector.setPort(8443);
    servers.get("admin").addConnector(adminSslConnector);

    Log.getRootLogger().info("CustomiseJettyServers - admin server info:");
    Log.getRootLogger().info("Admin SSL Exclude Cipher Suites:"+sslContextFactory.getExcludeCipherSuites());
    Log.getRootLogger().info("Admin SSL Include Cipher Suites:"+sslContextFactory.getIncludeCipherSuites());
    Log.getRootLogger().info("Admin Port:"+adminSslConnector.getPort());
  
    return servers;
}

The code above will allow you to set the following up:

• Custom keystore and truststore:

def SslContextFactory createSslContextFactory(String keyStorePath, String keyStorePassword, String trustStorePath,String trustStorePassword,String[] excludeProtocols,String[] includeCipherSuites) {

    SslContextFactory sslContextFactory = new SslContextFactory();
    sslContextFactory.setKeyStorePath(keyStorePath);
    sslContextFactory.setKeyStorePassword(keyStorePassword);
    sslContextFactory.setTrustStorePath(trustStorePath);
    sslContextFactory.setTrustStorePassword(trustStorePassword);

    sslContextFactory.addExcludeProtocols(excludeProtocols);
    sslContextFactory.setIncludeCipherSuites(includeCipherSuites);

    return sslContextFactory;
} 
:
:
:
SslContextFactory sslContextFactory = createSslContextFactory(
              <$SEARCH_HOME>/web/conf/keystore","OBF:<keystore password>",
              <$SEARCH_HOME>/web/conf/keystore (** truststore **)","OBF:<trustore password>",
              excludeProtocols,includeCipherSuites);

• Custom cipher suites and protocols:

:
:
    String[] excludeProtocols = ["SSL","SSLv2","SSLv2Hello","SSLv3"];
    String[] includeCipherSuites = ["TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384","TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256","TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384","TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256","TLS_DHE_RSA_WITH_AES_256_GCM_SHA384","TLS_DHE_RSA_WITH_AES_128_GCM_SHA256","TLS_DHE_RSA_WITH_AES_256_CBC_SHA256","TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA","TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA","TLS_DHE_RSA_WITH_AES_256_CBC_SHA","TLS_RSA_WITH_AES_256_GCM_SHA384","TLS_RSA_WITH_AES_128_GCM_SHA256","TLS_RSA_WITH_AES_256_CBC_SHA256","TLS_RSA_WITH_AES_128_CBC_SHA256","TLS_RSA_WITH_AES_256_CBC_SHA"];
:
:

To ensure that ports don't conflict on Jetty startup, edit $SEARCH_HOME/conf/global.cfg and remove off values for jetty.search_port, jetty.search_port_https and  jetty.admin_port so that the look like:

jetty.search_port=
jetty.search_port_https=
jetty.admin_port=

Listed below is an example global.cfg with notes on what to enter:

mail.smtp.host=localhost
mail.smtp.port=25
mail.smtp.auth=false
platform.bitness=64bit
mail.smtp.user=
mail.smtp.password=
jetty.search_port=
jetty.search_port_https= 
jetty.admin_port=
:
:
urls.search_port=<set to the same port as jetty.search_port >
urls.admin_port=<set to the CUSTOM admin port number>
:
:

Once $SEARCH_HOME/web/conf/customiseJettyServers.groovy and global.cfg have been edited, restart the jetty service.

Check the $SEARCH_HOME/log/jetty.log.* files for progress and for any errors.

Troubleshooting

jetty-logging.properties

Sometimes more detail can be gleaned by increasing the logging level found in Jetty.  

• Set logging levels for $SEARCH_HOME/log/jetty.log*  from  $SEARCH_HOME/web/conf/jetty-logging.properties
  • By default the logging level is set to INFO, but as its using java.util.logging.FileHandler as the base logger, use the following level settings (Source: https://docs.oracle.com/javase/8/docs/api/java/util/logging/Level.html)
    
    • SEVERE (highest value)
    • WARNING
    • INFO
    • CONFIG
    • FINE 
    • FINER
    • FINEST (lowest value)
  • In addition there is a level OFF that can be used to turn off logging, and a level ALL that can be used to enable logging of all messages.
• Listed below is an example of jetty-logging.properties set to FINE level.

handlers=java.util.logging.FileHandler

java.util.logging.FileHandler.level=FINE
# Working dir is tools/jetty/ (overwritten by the launcher anyway)
java.util.logging.FileHandler.pattern=../../log/jetty.log

# Write 10MB before rotating this file
java.util.logging.FileHandler.limit=10000000

# Append when file already exists
java.util.logging.FileHandler.append=true

# Number of rotating files to be used
java.util.logging.FileHandler.count=10
java.util.logging.FileHandler.formatter=java.util.logging.SimpleFormatter

.level=FINE

General Tips

• Look for tell-tale signs of syntax errors emanating from customiseJettyServers.groovy in the jetty logs.  If a syntax error is detected in the customiseJettyServers.groovy script, it WILL NOT EXECUTE.
• If customiseJettyServers.groovy contains code, the following will be seen in the $SEARCH_HOME/log/jetty.log*:
  • SEVERE: *** Server config has been altered with /opt/funnelback/web/conf/customiseJettyServers.groovy ***
  • Syntax errors in customiseJettyServers.groovy (if they exist) will appear below the above message in quite close proximity.
• Set logging messages from within the customiseJettyServers.groovy script.  Examples can be seen in the sample script above e.g.

    Log.getRootLogger().info("CustomiseJettyServers - public server info:");
    Log.getRootLogger().info("Public SSL Exclude Cipher Suites:"+sslContextFactory.getExcludeCipherSuites());
    Log.getRootLogger().info("Public SSL Include Cipher Suites:"+sslContextFactory.getIncludeCipherSuites());
    Log.getRootLogger().info("Public Port:"+publicSslConnector.getPort());