Apache Slider App Package

All application artifacts, app definition, app configuration, scripts are packaged into a structured single package that can be handed off to any YARN application deployment tool including Slider

Overall Structure

App package is a zip package containing all application artifacts. App package contains the following items:

  • app definition file application structure, content, definition, supported platforms, version, etc.

  • default configurations folder various configurations and configuration files associated with the application

  • cmd_impl folder management operations for the application/component

  • scripts folder various scripts that implement management operations

  • templates folder various templates used by the application

  • files folder other scripts, txt files, tarballs, etc.


The example above shows a semi-expanded view of an application "HBASE-YARN-APP" and the package structure for OOZIE command scripts.

app definition

App definition is a file named "metainfo.xml". The file contains application definition as described in Application Definition.

default configurations

This folder consists of various config files containing default configuration as described in App Configuration.

package folder

package includes the "implementation" of all management operations. The folders within are divided into scripts, templates, and files.

scripts folder

Scripts are the implementation of management operations. There are five default operations and a composite operation. "restart" can be redefined to have a custom implementation.

  1. install

  2. configure

  3. start

  4. stop

  5. status

  6. restart (by default calls stop + start)

The script specified in the metainfo is expected to understand the command. It can choose to call other scripts based on how the application author organizes the code base. For example:

class HbaseMaster(Script):
  def install(self, env):

  def configure(self, env):
    import params


  def start(self, env):
    import params
    self.configure(env) # for security

    hbase_service( 'master',
      action = 'start'

How to write scripts discusses how the scripts are developed and the structure of the JSON_FILE containing the parameters.

templates folder

templates are configurable text files that are NOT regular config files. A library has been developed that can materialize a complete site configuration file from a property bag and therefore are not generated from templates. Other files such as env sh files, log4j properties file, etc. may be derived from a template. Again, the implementor can choose to create these files from scratch and not use templates. The following operations are allowed during template expansion:

  • variable expansion

  • if condition

  • for loop

  • ...

Sample template file for dfs.exclude file to list excluded/decommissioned hosts. hdfs_exclude_files in the property defined in params.py which is populated from config parameters defined in JSON_FILE.

{% if hdfs_exclude_file %} 
{% for host in hdfs_exclude_file %}
{% endfor %}
{% endif %}

files folder

files is a directory to store any other files that are needed for management operations. Sample files stored here are tarballs used to install the application, shell scripts used by various operations.