Create a failover administration server using mediator and WedDAV

Managed by | Updated .

To setup a failover admin server configure your Funnelback environments as detailed.

Primary administration server

Configure the primary administration server as per the instructions on the Funnelback documentation website for configuring multiple query processors.

The following additional steps should be configured:

Disable the WebDAV service 

Edit global.cfg and add the following line:

$SEARCH_HOME/conf/global.cfg
daemon.services=GenerateCertificates,StoreService,FilterService

After changing this setting the Funnelback daemon will need restarting. Disabling WebDAV will prevent any other server pushing content to the admin server.

Check query processors are listed in config

Ensure the query processors and failover admin servers are listed as query processors in global.cfg.

$SEARCH_HOME/conf/global.cfg
query_processors=qp1.mycluster.com,qp2.mycluster.com,ad2.mycluster.com

Configure periodic sync

Create a sync_ad.groovy script to do a periodic sync of folders from primary admin to failover admin.  The following code can be used.  

The following script will recursively copy all the folders that are passed to it as arguments.  The script could be improved by making it walk the file system and push each file separately (adding support for include/exclude rules).  At the moment it's only possible to exclude files from the root folders that are copied.

$SEARCH_HOME/conf/sync_ad.groovy
import com.funnelback.common.config.*
import com.funnelback.mediator.core.webdav.*
import java.nio.file.*
/*
 * Command line tool to transfer arbitrary files to a remote Funnelback server
 * using the WebDAV protocol. It assumes that:
 *
 * - Both Funnelback instances share the same server_secret in global.cfg
 * - The WebDAV service is up and running on the remote side
 * - Access to the WebDAV port (usually 8076) has been opened in the firewall
 *
 * Only files from within the local Funnelback install can be transferred.
 *
 * $Id: transfer-webdav.groovy 31958 2012-10-12 05:06:50Z nguillaumin $
 */
// Port of the remote WebDAV service
def final DEFAULT_WEBDAV_PORT = 8076;
// Setup command line options
def cli = new CliBuilder(usage: getClass().simpleName+' -h <remote_host> <file or folder> [file or folder] ...')
cli.with {
    header = "Transfer a file to a remote Funnelback server.\n" + "Both Funnnelback servers must have the same server_secret"
    h longOpt: "host", argName: 'hostname', args:1, type: String.class, "Remote host name"
    p longOpt: "port", argName: 'port', args:1, type: Number.class, "Remote host WebDAV port (Defaults to $DEFAULT_WEBDAV_PORT)"
    d longOpt: "delete", "Delete remote files that don't exist locally (Default: false)"
    s longOpt: "smart", "Do not transfer file that already exist remotely, based on the filesize (Default: false)"
    r longOpt: "recursive", "Transfer folders recursively (Default: false)"
    v longOpt: "verbose", "Verbose mode"
}
def options = cli.parse(args)
if (!options) return;
//if (!options.h || !options.arguments()) {
if (!options.h) {
    // No host name, or no file to transfer
    cli.usage()
    return;
}
def port = DEFAULT_WEBDAV_PORT
if (options.p) port = options.p
def fs = FileSystems.getDefault()
def searchHome = fs.getPath(new File(System.getenv("SEARCH_HOME")).absolutePath)
def config = new GlobalOnlyConfig(searchHome.toFile())               // Global config data, for the server_secret
def webdav = new FunnelbackSardine(config.value("server_secret"))    // WebDAV client, to push files
def urlHelper = new WebDAVUrlFactory(options.'host', port)           // Helper to build URL to the remote machine
def seeds = options.arguments()
//def seeds = ["d:\\funnelback\\conf\\","d:\\funnelback\\admin\\","d:\\funnelback\\databases\\"]
// For each file or folder...
//options.arguments().each {
seeds.each() {
    def seed = new File(it)
    files=seed.listFiles()
    files.each() {
        // The actual file
        def file=it
        //def file = new File(it)
        // A path object makes it easer to manipulate
        def path = fs.getPath(file.absolutePath)
        if (! file.exists()) {
            println "Skipping '$path' as it doesn't exist"
        } else if (file =~ /global\.cfg$/) {
            println "Skipping '$path' as it's global.cfg"
        } else if (! path.startsWith(searchHome)) {
            println "Skipping '$path' as it doesn't belong to SEARCH_HOME ($searchHome)"
        } else {
            def targetUrl = urlHelper.root() + searchHome.relativize(path).toString().replace("\\", "/")
            println "Transferring '$path' to '$targetUrl'"
            if (file.isDirectory()) {
                def transferred;
                if (options.r) {
                    transferred = webdav.recursiveSync(
                        targetUrl,
                        file,
                        options.d,
                        options.s);
                } else {
                    transferred = webdav.sync(
                        targetUrl,
                        file,
                        options.d,
                        options.s);
                }
                // Display transferred files
                if (options.v) {
                    println "${transferred.size} files transferred"
                    transferred.each {
                        System.out << ((it.success) ? "OK   " : "FAIL ")
                        System.out << it.filename.padRight(40) << " "
                        System.out << ((it.success) ? "" : it.statusCode.toString())
                        System.out << ((it.errorMessage) ? " " + it.errorMessage : "")
                        System.out << "\n"
                    }
                }
            } else {
                webdav.put(targetUrl, file);
            }
        }
    }
}

Create a wrapper script to call the groovy code and sync $SEARCH_HOME/conf/, $SEARCH_HOME/admin/ and $SEARCH_HOME/databases/.  

The script does not copy global.cfg which should not be overwritten. 

Under Windows the following script can be used to call the groovy code above:

$SEARCH_HOME/conf/sync_ad.bat
@ECHO off
REM Command to configuration to failover admin server.  Runs every 10 minutes via a scheduled task.
REM USAGE: sync_ad.bat <failover admin host>
IF "%~1"=="" (
@ECHO Usage: sync_ad.bat ^<failover_admin_host^>
EXIT /B 1
) ELSE IF ""=="" (
@ECHO ERROR: SEARCH_HOME is not defined
EXIT /B 1
) ELSE (
SET JAVA_HOME=\wbin\java
\tools\groovy\bin\groovy -cp "\lib\java\all\funnelback-common.jar;\lib\java\all\funnelback-mediator-core.jar;\lib\java\all\sardine-314.jar;\lib\java\all\log4j-1.2.16.jar;\lib\java\all\jedis-2.1.0.jar;\lib\java\all\commons-pool-1.5.5.jar;\lib\java\all\commons-io-2.4.jar;\lib\java\all\httpclient-4.3.2.jar;\lib\java\all\httpcore-4.3.2.jar;\lib\java\all\commons-collections-3.2.jar;\lib\java\all\slf4j-log4j12-1.6.4.jar;\lib\java\all\slf4j-api-1.6.4.jar;\lib\java\all\commons-codec-1.9.jar" \conf\sync_ad.groovy -h %1 -s -d -v -r \conf\ \admin\ \databases\
)

Schedule periodic sync

Create a scheduled task / cron job to run the groovy script every 10 minutes.  To call the script above execute $SEARCH_HOME/conf/sync_ad.bat <failover_admin_server> and schedule this for a 10 minute update cycle.

Failover administration server

The following instructions cover server configuration for a failover administration server that provides limited failover services:

  • Read only access to server configuration
  • Access to analytics
  • Updates are disabled

Configure the failover administration server as per the instructions for configuring a query processor server on the above link.

The following additional steps should be configured:

  1. Ensure the server shares the same server secret as other servers in the cluster
  2. Ensure the WebDAV port is configured correctly (default is port 8076)
  3. Configure the administration UI to operate in read only mode:
$SEARCH_HOME/conf/global.cfg
admin.read-only-mode=true
  1. Disable scheduled tasks/cron jobs for Funnelback related updates (Update analytics, Update outliers)      
  2. (optional) Configure load balancer rules as appropriate to failover users to this server.      

Convert failover administration server to the primary admin server

The following instructions can be used to convert a failover admin server into the primary admin server:

If the old primary admin server is still available do the following:

  1. Stop any running updates.   
  2. Disable scheduled tasks/crob jobs (Sync configuration, collection updates, report updates, outliers updates)      
  3. Enable Admin UI read only mode.    
  4. Run the sync configuration scheduled task manually to ensure failover administration has the latest configuration.
  5. Comment out the query_processors line in global.cfg.

Convert the failover admin server to the primary admin server

  1. Disable Admin UI read only mode.    
  2. Disable the WebDAV service.
  3. Configure the query processors.
  4. Schedule collection updates, updates of reports/outliers.
  5. (optional) Setup and configure a sync configuration task to clone configuration to any other failover administration server.
Was this artcle helpful?

Comments