3. General usage¶

Commandline interface¶

Generally, the EBAS Commandline tools are used within a commandline interpreter - usually a Linux shell, but the commandline interpreter in Windows(R) (cmd.exe, command prompt, aka MS-DOS prompt) is also a possible option.

For the scope of this document we limit the explanations to the Linux shell, although the concepts are general and work similar in alternative commandline interpreters.

How to run an EBAS program¶

The EBAS Commandline programs are all started from the commandline interface (Linux shell) by typing the program name and adding commandline arguments to control what the program should do. The program name and all arguments are separated by (at least) one blank character.

Example:

ebas_insert NO0001R.20100101000000.20120817103603.aerosol_light_backscattering_coefficient_statistics.pm10.1y.1h.nephelometer.NO01L_TSI3563_70810508.NO01L_scat_coef.nas

In this example we see the program name ebas_insert (insert data into the database) and a commandline argument NO0001R.20100101000000.20120817103603.aerosol_light_backscattering_coefficient_statistics.pm10.1y.1h.nephelometer.NO01L_TSI3563_70810508.NO01L_scat_coef.nas separated by a blank.

Commandline arguments¶

Nearly all functionality of the EBAS programs is controlled by commandline arguments, so that the program can run without further input from the user (there are some rare exceptions to this rule, this is explained in the documentation of the respective programs).

Commandline arguments can be mandatory (they must be specified) or optional (they can be specified, but do not need to be).

Optional arguments¶

Optional arguments start with two hyphens (--) followed by a keyword, e.g. --help. The most frequently used arguments have an alternative short form with one hyphen followed by one single character, e.g. -h. Note that there is no difference between using the short or the long form, it’s up to personal preferences. People using a specific program very frequent generally prefer to use the short form for typing speed. People using a program rather seldom may find the long form easier to remember.

Some optional arguments are mere switches and do not require additional information. Those arguments are called options. An Example for an option is --help (-h): The user either requests help (passes the argument --help) or not. No value is required for help.

Other optional arguments arguments require associated values to be passed, e.g. the argument --setkey (-s) requires one or more dataset setkeys. The argument name and value(s) can either be separated by a blank character or an equal sign. The following 4 arguments are equivalent: -s 9 --setkey 9 -s=9 --setkey=9.

Mandatory arguments¶

Mandatory arguments have no keyword, but just a value. The argument is defined by it’s position only (this is only possible because the argument is mandatory).

Mandatory arguments can in principle be specified at any position in the commandline, as long as the order of the mandatory arguments is correct. However, putting the mandatory arguments all in the end of the commandline (after optional arguments and their values) is not only good custom, but first and foremost improves readability and avoids confusion.

Subcommands¶

Subcommands are used to split the behaviour of the program into some major directions. Subcommands are used very rarely in EBAS, one example are the action subcommands of ebas_set_project.

It is important to understand the nature of subcommands. They can be understood as mandatory keywords, additionally they are mutually exclusive. That means, one and only one subcommand must be specified.

Unlike other arguments, the order of arguments is important when using subcommands. After specifying a subcommand name, only sub-arguments for this subcommand are valid. All other arguments must be specified before the subcommand.

Program output conventions¶

Data output¶

By default the data output of all EBAS programs is written to the commandline terminal, to be specific to the channel stdout. The definition of data output depends on the type of program. For example, for ebas_list_ds the output is the list of dataset metadata, for ebas_extract, the output is the data extraction.

Not all programs do produce data output. Some programs do not produce any output by definition (e.g. ebas_delete or ref:EBASprogram_ebas_insert). Others produce output only in some cases: ebas_extract for example has an option ebas_extract --createfiles to create output files instead of writing to stdout (if this option is turned on, ebas_extract is rendered to a program without any data output).

Logging output¶

Logging information (meta information about the program run) is by default also written to the commandline terminal, but to a different channel stderr. Additionally is logging information marked with the logging severity in the beginning of each line. This way it can be distinguished from data output. Logging severities are for example INFO, WARNING and ERROR.

Examples for data and log output:

$ ebas_list_ds --setkey 9
INFO    : number of datasets: 1
     9 CA0420G high_vol_sampler                         sulphate_total                                                                                                                               aerosol                                  CA01L_Rahn1                              CA01L_IC1                                1w    1992-12-28T00:00:00  1993-06-29T00:00:00 AMAP,AMAP_public

Here we can see one line of logging information: INFO : number of datasets: 1, which is meta information about the program run. The second line of output is data output from ebas_list_ds.

Controlling logging information verbosity¶

The output (or prevention) of log messages can be controlled by the argument --loglevconsole for all EBAS commandline programs.

If you want to have logging information written to log files (additionally or instead of stdout), have a look at the arguments --loglevfile and --logfile.

Finding EBAS Commandline programs¶

For getting a quick list of available EBAS programs, you can use the commandline completion option of your shell. Most commonly, shells use the tab key to complete, so we use tab in the example below:

$ ebas_<tab>
ebas_aot           ebas_delete        ebas_diurnal       ebas_finnmaks      ebas_geom_stdev    ebas_list_comp     ebas_list_ds       ebas_list_org      ebas_max           ebas_min           ebas_num_samples   ebas_stdev
ebas_capture       ebas_dep           ebas_extract       ebas_geom_mean     ebas_insert        ebas_list_data     ebas_list_meta     ebas_list_station  ebas_mean          ebas_num_bdl       ebas_perc          ebas_sum

Online help function¶

All EBAS programs have a commandline argument --help which will display a short description of the purpose of the program and describe all possible commandline arguments for the program.

Configuration file¶

Configuration files can be used to change the behaviour or EBAS Commandline programs. As a default, a file named ebas.cfg in the user’s home directory will be used as configuration file. Using the commandline argument --cfgfile when executing an EBAS commandline program, a different file can be specified.

Configuration files can be used to set a default for any commandline argument in all EBAS commandline programs. The names of all configuration file settings correspond to the commandline argument names (without the -- prefix). The only commandline argument which does not make sense to set in a configuration file is --cfgfile itself.

Configuration file settings are used as a default for the respective commandline argument whenever a program is executed. Configuration settings override the system default values, but can be overridden at every individual program run by specifying commandline arguments.

Structure of the configuration file¶

The EBAS configuration file generally follows the INI file format definition.

The file uses one [EBAS] section for setting global configurations for all EBAS programs. The [EBAS] section is mandatory.

There might be additional sections for configuring specific programs. The respective program name is used as a section name (e.g [ebas_list_ds] for the configuration settings for ebas_list_ds).

Whole lines may be commented out using the # or ; characters. For inline comments, only ; can be used. See the example below.

Example cofiguration file:¶

[EBAS]
loglevfile=info   ; a semicolon marks an inline comment
# for whole line comments, a numbersign
; or a semicolon may be used
dbHost=DEVSYBASE  ; use the dev DB by default (because I'm mostly testing and not working operational)

[ebas_list_ds]
loglevfile=debug
loglevconsole=debug

The configuration in this example:

Sets the log level for logfile output to info for all EBAS programs, which means a logfile will be written for each EBAS program run.

Sets the database host to DEVSYBASE for all EBAS programs, which means, all programs will connect to the test database server by default, handy for software developers and testers.

The loglevel for log file output is set to debug for the program ebas_list_ds), which will produce very big logfiles each time this program is executed.

The log level for console output (commandline terminal) is set to debug as well for the program ebas_list_ds), which will produce very detailed log information in the terminal output each time this program is executed.

Remember, each of the configuration file settings can be overridden at program runs by specifying the respective commandline argument with the same name.

Example cofiguration file for a data manager:¶

For a data manager, working with operational dataflows, the following configuration may be a good starting point:

[EBAS]

[ebas_insert]
loglevfile=info   ; always create a logfile when inserting data
email_notify=True  ; send emails to data submitter when data have been inserted
email_bcc=me@nilu.no  ; send blind copy to myself, to keep track of emails sent to data submitters. Substitute me@nilu.no with your emial address!