|
|
ldasJob { -name {} -password {} -email {} } { userCmd -opt1 {} ... }
Which is in the format of a Tcl command, ldasJob, with two required arguments:
The putStandAlone user command is the "back end" of the dataStandAlone user command. Wrapper API output data from a dataStandAlone command is read from disk and passed to the eventmon API for ingestion into the database tables.
Calling convention (all on a single line):
ldasJob { -name "username" -password "********" -email "user@foobar.edu"} {putStandAlone -wrapperdata URL -returnprotocol URL -metadataapi URL -database dsname -outputformat {ilwd ascii | ilwd binary | LIGO_LW } -subject {freeform string} }
Option Descriptions:
-wrapperdata:
http ftp file
NOTES:
-returnprotocol:
http ftp gridftp mailto file port
Default Value: http://results.ilwd
Note that in most cases this option should NOT be explicitly
set by the user, since it can have unexpected interactions
with the output() actions in the datacond API, for example.
The argument to -returnprotocol resembles the usual
browser conventions for URL's, and is used to control the
naming and disposition of output data from a user request.
The http://, ftp://, gridftp:/,
and file:/ URI types are currently supported.
The specific format of the argument, and possibly the values
of other arguments will affect the way that the
-returnprotocol argument is applied by the LDAS system:
Will cause the system to return the location of output
data as an ftp or http URL (the system default, in the
absence of a user supplied -returnprotocol option, is
to use http).
Will cause the system to return the location of output
data as an ftp or http URL (the system default, in the
absence of a user supplied -returnprotocol option, is
to use http). And any output object which is not
in frame format will be named according to the
suggested pattern. When possible, indexes will be
appended to the filename to differentiate between multiple
output files, but sometimes this can not be managed and in
that case as many levels of .ba* files will be created as
required to avoid overwriting data.
When this form is used and dirname corresponds to an
existing directory in the anonymous ftp area of the LDAS
system, all output files will be copied into that directory.
In some cases a filename can be specified and will be used;
in this case .ba* files may be created to avoid overwriting
of data. This option of -returnprotocol is currently is not supported. The output
e.g. frame or xml, is placed in the job output directory.
The user has to retrieve the output via http or ftp to
to his/her local system.
When this form is used the LDAS system will attempt to copy
the output data via anonymous ftp to the remote site. This option of -returnprotocol is currently is not supported. The output
e.g. frame or xml, is placed in the job output directory.
The user has to retrieve the output via http or ftp to
to his/her local system.
This is a special protocol used by the dataStandAlone user
command for interaction with a GLOBUS/GRID system.
This returnprotocol is only interpreted by the datacond API
for use on output data formatted for ingestion by a stand alone
wrapper API.
The file will be written to the local gridftp home directory
or a subdirectory of it which is writable by the user which
LDAS runs as.
Several modified forms of the gridftp URL are available:
as a useful shorthand notation for producing a directory
hierarchy of dataStandAlone results which maps directly to
the job directories normally produced by LDAS, allowing
simple correlation of output data with other job resuts or
messages from the LDAS system.
Then the directory hierarchy will be created as required
and unique job-specific default filenames will be assigned
to the datacond API's output for the wrapper, i.e.:
NOTES:
-outputformat: frame ilwd LIGO_LW
The argument of the -outputformat option is the data type to
use in formatting the result of the user request.
-metadataapi: api
Default: metadata
The metadata results will be inserted into the database.
If 'ligolw' is specified e.g. -metadataapi ligolw,
and the -returnprotocol specifies a url, the metadata results will be output in the form of xml
rather than being inserted into the database i.e.
running at a site that does not have a database.
If 'tee' option is specified, e.g. -metadataapi tee, and the -returnprotocol specifies a url
the metadata results will
be output both as an xml and also be inserted
into the database.
Examples of putStandAlone commands:
Examples on how to use this command
This example presents a Tcl script which can be run from any
computer with a network connection to submit a user command
to the LDAS system:
Example of xml output
using the default option for -metadataapi, the response is database insertion:
If '-metadataapi tee' option is used, the response is both xml and database output:
The argument of the -wrapperdata option URL format
is of the format similar to
that of -returnprotocol. The URL specifies the source of
of the data to be ingested in ilwd format: it can be files,
or directory containing wrapper files.
The URL must be valid for the command to be successfully executed.
Optional
Specifying "file" will return the local path to the file
as it is seen by all LDAS API's.
Specifying ftp or http will return a URL
relative to the gateway machine of the local system (which
is the same machine to which user requests are made).
The names used for the output files will be determined
by the system and are generally descriptive of the
content of the files.
The "file" form is used to get the local name of the file
relative to the LDAS installation for use as input data
by subsequent jobs.
This form is not advised when the job is one that is expected
to produce many output files!
In some special cases the suggested filenamr and extension
may be ignored, as when a frame file is being produced, or
an exact ilwd representation of a frame file.
Note that the trailing slash is absolutely required in order
for the URL to be interpreted as a local directory by the
LDAS system.
The URL returned to the user will be of the type specified.
The user will need to run tests to make certain that this
form works with the specified site.
The LDAS writable subdirectory should be defined in the
LDASapi.rsc resource file as the
::GRID_FTP_WRITABLE_SUBDIRECTORY, which will be joined
to the grid home directory when calculating output filenames.
-returnprotocol gridftp
-returnprotocol gridftp://here/there/mydir/mysubdir/
LDAS-TEST1234567_wrapperdata.ilwd
Default: {ilwd binary}
The possible output formats which the system can produce include:
#!/bin/sh
# use -*-Tcl-*- \
exec tclsh "$0" "$@"
# convenient formatted command definition
# database insertion from wrapper files
set cmd "ldasJob
{ -name foo -password **** -email foo@foobar.edu }
{
putStandAlone
# subject to be used when e-mailing user
-subject {test script for putStandAlone}
# input url for converting
-wrappperdata file:/ldas_outgoing/jobs/globus_files/NORMAL1548
# ilwd format for output ilwd (optional)
-outputformat {ilwd ascii}
# database if not default
-database ldas_tst
}"
# strip comments and compact cmd into user command format
set cmd [ split $cmd "\n" ]
foreach line $cmd {
set line [ string trim $line ]
if { [ regexp {^#} $line ] } {
continue
}
lappend tmp $line
}
set cmd [ join $tmp "\n" ]
regsub -all -- {[\n\s]+} $cmd { } cmd
# connect, issue command, print reply from manager,
# and disconnect
set sid [ socket ldas-dev.ligo.caltech.edu 10001 ]
puts $sid $cmd
flush $sid
puts [ read $sid ]
close $sid
Which submits this command to the system:
# xml output from wrapper files
set cmd "ldasJob
{ -name foo -password **** -email foo@foobar.edu }
{
putStandAlone
# subject to be used when e-mailing user
-subject {test script for putStandAlone}
# input url for converting
-wrappperdata file:/ldas_outgoing/jobs/globus_files/NORMAL1548
-returnprotocol http://results
-outputformat {LIGO_LW}
-metadataapi { ligolw }
}"
Output will be created by ligolw API in job output directory:
Your results:
results.xml
can be found at:
http://131.215.115.248/ldas_outgoing/jobs/LDAS-DEV_1049/LDAS-DEV10494929
Subject: BOX-III98872 tfclu online running on H2:LSC-AS_Q Inserted
1 rows into ldas_tst database table process Inserted 14 rows into ldas_tst database
table process_params Inserted 1330 rows into ldas_tst database table sngl_burst
Inserted 1 rows into ldas_tst database table search_summary
Subject: BOX-III98809 tfclu online running on H2:LSC-AS_Q Inserted 1 rows into ldas_tst database
table process Inserted 14 rows into ldas_tst database table process_params
Inserted 1330 rows into ldas_tst database table sngl_burst Inserted 1 rows into ldas_tst database
table search_summary Your results: tfclusters_result.xml can be found
at: http://131.215.114.23/ldas_outgoing/jobs/BOX-III_9/BOX-III98809
Return to Top