*************** General usage *************** .. _EbasCommandlineInterface: Commandline interface ===================== Generally, the EBAS Commandline tools are used within a commandline interpreter - usually a Linux shell, but the commandline interpreter in Windows(R) (cmd.exe, command prompt, aka *MS-DOS prompt*) is also a possible option. For the scope of this document we limit the explanations to the Linux shell, although the concepts are general and work similar in alternative commandline interpreters. How to run an EBAS program -------------------------- The EBAS Commandline programs are all started from the commandline interface (Linux shell) by typing the *program name* and adding commandline arguments to control what the program should do. The program name and all arguments are separated by (at least) one blank character. Example:: ebas_insert NO0001R.20100101000000.20120817103603.aerosol_light_backscattering_coefficient_statistics.pm10.1y.1h.nephelometer.NO01L_TSI3563_70810508.NO01L_scat_coef.nas In this example we see the *program name* ``ebas_insert`` (insert data into the database) and a *commandline argument* ``NO0001R.20100101000000.20120817103603.aerosol_light_backscattering_coefficient_statistics.pm10.1y.1h.nephelometer.NO01L_TSI3563_70810508.NO01L_scat_coef.nas`` separated by a blank. .. _EBASCommandlineArguments: Commandline arguments --------------------- Nearly all functionality of the EBAS programs is controlled by *commandline arguments*, so that the program can run without further input from the user (there are some rare exceptions to this rule, this is explained in the documentation of the respective programs). Commandline arguments can be mandatory (they must be specified) or optional (they can be specified, but do not need to be). Optional arguments ^^^^^^^^^^^^^^^^^^ Optional arguments start with two hyphens (``--``) followed by a keyword, e.g. :option:`--help`. The most frequently used arguments have an alternative *short form* with one hyphen followed by one single character, e.g. :option:`-h`. Note that there is no difference between using the short or the long form, it's up to personal preferences. People using a specific program very frequent generally prefer to use the short form for typing speed. People using a program rather seldom may find the long form easier to remember. Some optional arguments are mere switches and do not require additional information. Those arguments are called *options*. An Example for an option is :option:`--help` (:option:`-h`): The user either requests help (passes the argument ``--help``) or not. No value is required for help. Other optional arguments arguments require *associated values* to be passed, e.g. the argument :option:`--setkey` (:option:`-s`) requires one or more :term:`dataset setkeys `. The argument name and value(s) can either be separated by a blank character or an equal sign. The following 4 arguments are equivalent: ``-s 9`` ``--setkey 9`` ``-s=9`` ``--setkey=9``. Mandatory arguments ^^^^^^^^^^^^^^^^^^^ Mandatory arguments have no keyword, but just a value. The argument is defined by it's position only (this is only possible because the argument is mandatory). Mandatory arguments can in principle be specified at any position in the commandline, as long as the *order of the mandatory arguments* is correct. However, putting the mandatory arguments all in the end of the commandline (after optional arguments and their values) is not only good custom, but first and foremost improves readability and avoids confusion. Subcommands ^^^^^^^^^^^ Subcommands are used to split the behaviour of the program into some major directions. Subcommands are used very rarely in EBAS, one example are the :ref:`action subcommands of ebas_set_project `. It is important to understand the nature of subcommands. They can be understood as mandatory keywords, additionally they are mutually exclusive. That means, one and only one subcommand must be specified. Unlike other arguments, the order of arguments is important when using subcommands. After specifying a subcommand name, only sub-arguments for this subcommand are valid. All other arguments must be specified before the subcommand. Program output conventions -------------------------- Data output ^^^^^^^^^^^ By default the *data output* of all EBAS programs is written to the commandline terminal, to be specific to the channel `stdout `_. The definition of data output depends on the type of program. For example, for :ref:`EBASprogram_ebas_list_ds` the output is the list of dataset metadata, for :ref:`EBASprogram_ebas_extract`, the output is the data extraction. Not all programs do produce data output. Some programs do not produce any output by definition (e.g. :ref:`EBASprogram_ebas_delete` or ref:`EBASprogram_ebas_insert`). Others produce output only in some cases: ``ebas_extract`` for example has an option :option:`ebas_extract --createfiles` to create output files instead of writing to ``stdout`` (if this option is turned on, ``ebas_extract`` is rendered to a program without any data output). Logging output ^^^^^^^^^^^^^^ Logging information (meta information about the program run) is by default also written to the commandline terminal, but to a different channel `stderr `_. Additionally is logging information marked with the logging severity in the beginning of each line. This way it can be distinguished from data output. Logging severities are for example INFO, WARNING and ERROR. Examples for data and log output:: $ ebas_list_ds --setkey 9 INFO : number of datasets: 1 9 CA0420G high_vol_sampler sulphate_total aerosol CA01L_Rahn1 CA01L_IC1 1w 1992-12-28T00:00:00 1993-06-29T00:00:00 AMAP,AMAP_public Here we can see one line of logging information: ``INFO : number of datasets: 1``, which is meta information about the program run. The second line of output is data output from ``ebas_list_ds``. Controlling logging information verbosity """"""""""""""""""""""""""""""""""""""""" The output (or prevention) of log messages can be controlled by the argument :option:`--loglevconsole` for all EBAS commandline programs. If you want to have logging information written to log files (additionally or instead of ``stdout``), have a look at the arguments :option:`--loglevfile` and :option:`--logfile`. Finding EBAS Commandline programs --------------------------------- For getting a quick list of available EBAS programs, you can use the `commandline completion option `_ of your shell. Most commonly, shells use the ``tab`` key to complete, so we use ``tab`` in the example below:: $ ebas_ ebas_aot ebas_delete ebas_diurnal ebas_finnmaks ebas_geom_stdev ebas_list_comp ebas_list_ds ebas_list_org ebas_max ebas_min ebas_num_samples ebas_stdev ebas_capture ebas_dep ebas_extract ebas_geom_mean ebas_insert ebas_list_data ebas_list_meta ebas_list_station ebas_mean ebas_num_bdl ebas_perc ebas_sum Online help function -------------------- All EBAS programs have a commandline argument :option:`--help` which will display a short description of the purpose of the program and describe all possible commandline arguments for the program. .. _EbasCommandlineConfigurationFile: Configuration file ================== Configuration files can be used to change the behaviour or EBAS Commandline programs. As a default, a file named ``ebas.cfg`` in the user's home directory will be used as configuration file. Using the commandline argument :option:`--cfgfile` when executing an EBAS commandline program, a different file can be specified. Configuration files can be used to set a default for any :doc:`commandline argument` in all EBAS commandline programs. The names of all configuration file settings correspond to the commandline argument names (without the ``--`` prefix). The only commandline argument which does not make sense to set in a configuration file is :option:`--cfgfile` itself. Configuration file settings are used as a default for the respective commandline argument whenever a program is executed. Configuration settings override the system default values, but can be overridden at every individual program run by specifying commandline arguments. Structure of the configuration file ----------------------------------- The EBAS configuration file generally follows the `INI `_ file format definition. The file uses one ``[EBAS]`` section for setting global configurations for all EBAS programs. The ``[EBAS]`` section is mandatory. There might be additional sections for configuring specific programs. The respective program name is used as a section name (e.g ``[ebas_list_ds]`` for the configuration settings for :ref:`EBASprogram_ebas_list_ds`). Whole lines may be commented out using the ``#`` or ``;`` characters. For inline comments, only ``;`` can be used. See the example below. Example cofiguration file: -------------------------- :: [EBAS4] loglevfile=info ; a semicolon marks an inline comment # for whole line comments, a numbersign ; or a semicolon may be used dbHost=dev-ebas-db01 ; use the dev DB server DON'T DO THIS IF YOU DON'T KNOW WHAT THAT MEANS! [ebas_list_ds] loglevfile=debug loglevconsole=debug The configuration in this example: * Sets the log level for logfile output to ``info`` for all EBAS programs, which means a logfile will be written for each EBAS program run. Handy for testing, but maybe not useful for a general user. * Sets the database host to the DEVELOPMENT SERVER for all EBAS programs, which means, all programs will connect to the test database server by default, handy for software developers and testers. DO NOT USE FOR USING EBAS DATA! * The loglevel for log file output is set to ``debug`` for the program :ref:`EBASprogram_ebas_list_ds`), which will produce very big logfiles each time this program is executed. * The log level for console output (commandline terminal) is set to ``debug`` as well for the program :ref:`EBASprogram_ebas_list_ds`), which will produce very detailed log information in the terminal output each time this program is executed. Remember, each of the configuration file settings can be overridden at program runs by specifying the respective commandline argument with the same name. Example cofiguration file for a data manager: --------------------------------------------- For a data manager, working with operational dataflows, the following configuration may be a good starting point: :: [EBAS4] [ebas_insert] loglevfile=info ; always create a logfile when inserting data The configuration in this example: * Sets the log level for logfile output to ``info`` for the program :ref:`EBASprogram_ebas_insert`. A log file with severity `info` and above is created for each run of this program.