NXM Help CONCEPTS
Contents
- 1 CONCEPTS - This module catalogs the concepts for NeXtMidas users and programmers.
- 2 COMMANDS
- 3 CommandLine - Covers command line syntax and processing.
- 4 Configuration - Adding and finding commands in an option tree.
- 5 Recall - Recalling and completing commands.
- 6 Completion - Completing commands.
- 7 Intrinsics - shell control and other immediate commands
- 8 Primitives - data handling and processing commands
- 9 Macros - the NeXtMidas scripting language commands
- 10 UserMacros - Creating and Running user macros.
- 11 OS - Executing Operating System commands
- 12 Switches - See Users's Guide -> System Operations -> Switches
- 13 FILES - Working with data files of various types.
- 14 FILES - NeXtMidas file system support
- 15 Auxes - How NeXtMidas finds files.
- 16 Groups - Definition of Midas file groups.
- 17 Resources - Definition of file resources
- 18 Types - Types of Midas Data files
- 19 TRIMMERS - Using trimmers to thin files in-line.
- 20 QUALIFIERS - Using qualifiers to modify file behavior.
- 21 QUALIFIERS - File qualifiers in NeXtMidas use curly braces, {}; trimmers use parentheses,
- 22 AUX - This short-circuits the normal search through the read aux list.
- 23 TAG - This sets the ID of the file for plot layers.
- 24 FLAGS - This sets optional flags to tailor the behavior of the file.
- 25 FS - This qualifier can be used to override the frame size of a file on-the-fly.
- 26 FG - The FG (File Group) qualifier can be used to specify the category of the file.
- 27 EXIST - The EXIST qualifier is used to tailor the existence algorithm used for a
- 28 TAG - The {TAG=name} qualifier may be used to set the "ID" of a file.
- 29 CFL - The Circular File Length qualifier sets the length in seconds of the disk file
- 30 TRIMMERS - Using trimmers to thin files in-line.
- 31 HELP - Help with this HELP engine
- 32 Commands - TREE - Show the help tree of topics/sections.
- 33 Usage - This section describes how to use help
- 34 Building_Help_Files - Help files are built by writing text as it should appear in a help browser.
- 35 or - closes on the previous section.
- 36 NotFound - This help section entry was not found.
- 37 PIPES - Memory files, writing "piped" macros.
- 38 SHELL - Using the NeXtMidas Shell, command syntax.
- 39 Switches - See Users's Guide -> System Operations -> Switches
- 40 RMIF - Remote Midas InterFace Protocol
- 41 RMIF - the Remote Midas InterFace protocol.
- 42 RMI - Java's Remote Method Invocation protocol.
- 43 Header - The RMIF 8-byte packet header.
- 44 AdjunctHeader - The RMIF adjunct header.
- 45 Functions - The currently defined RMIF functions.
- 46 Methods - How RMIF packets are converted to function calls.
- 47 RMIvsRMIF - Why RMIF and UDP instead of RMI and TCP.
- 48 MFTP - The Midas File Transfer Protocol
CONCEPTS - This module catalogs the concepts for NeXtMidas users and programmers.
This module catalogs the concepts for NeXtMidas users and programmers.
COMMANDS
Installing, finding and running commands.
There are a number of types of commands which may be run from the NextMidas shell:
- Command Line Processing
- Primitives, Intrinsics, Macros
- User Macros
- Operating System Commands
Commands are CASE-INSENSITIVE and may be abbreviated depending on how they are entered in the configuration file.
CommandLine - Covers command line syntax and processing.
Command line parameters and switches are specified using a key=value syntax. If the key= is omitted, the key is obtained from the command dictionary by its positionally dependent (PD) index. Only PD args are counted when determining the PD index. Key=value or /key=value entries are ignored. See the examples below.
Switches are normally placed directly after the command name before the 1st delimiter. They may also appear in a parameter slot but do not affect the PD index mapping. Switches are specified by preceding the name with a "/" and must be fully specified. Parameter names, if configured in the dictionary entry, may be abbreviated.
The PLOT command, for example, has the command dictionary entry: PLOT,P,8,FILE,SM=LINEAR,CM=MAG,LINE=SOLID,X1,X2,Y1,Y2 The command line: PLOT/id=psdplot/tl=1k input y1=-100 /name="My Plot" y2=200 /bg creates an arguments Table with the values:
/ID=PSDPLOT /TL=3 /NAME=My Plot FILE=INPUT SM=LINEAR CM=MAG LINE=SOLID X1= X2= Y1=-100 Y2=200 /BG=1
To access the arguments from inside a command:
filename = MA.getS("FILE"); tl = MA.getL("/TL"); or tl = MA.getL("/TL",-1); ymin = MA.getD("Y1"); or ymin = MA.getD("Y1",0.0);
Note that state switches (no =value) must appear directly after the command name or at the end of all parameters. If a state switch appears in a parameter list, it is treated as the "value of" that positionally dependent parameter. This allows a more natural syntax for branching on switches applied to a macro from within the macro. For instance, if the SD360 macro is run with the /CLIENT switch, a branch within the macro can be written as:
if /CLIENT gt 0 ! do this endif
To force the shell to treat an entry starting with a / from being treated as a switch anywhere in the parameter list, escape it with another /. For instance, to write the above branch without the "gt 0" so the test defaults to > 0, the /CLIENT parameter must be escaped to prevent it from being treated as a trailing switch.
if //CLIENT ! do this endif
For performance reasons, command lines are parsed only once when a macro is loaded. To allow runtime command line manipulation, carets are evaluated when a parameter is retrieved. This helps macros execute fast while still giving some level of macro parameter manipulation at runtime. The down side is that multiple arguments or switches cannot be stored in a caret'ed results parameter and parsed as one can from X-Midas. The /args=myargs switch can be used to generate a runtime-dependent list of parameters for a command in a macro. The switch should point to a table in the results table that has entries to be added to the currently parsed list. For instance,
if /client gt 0 set myargs.x1 100 set myargs.x2 200 else set myargs.y1 100 set myargs.y2 200 endif plot myfile /args=myargs
Configuration - Adding and finding commands in an option tree.
Adding and finding commands in an option tree.
The following commands may be used to configure and list commands. UPDATE - to configure a command in an option tree MENU - to display a list of commands in an option tree
Recall - Recalling and completing commands.
Commands may be recalled by using the bang (!) symbol. For example, the command
nm> status myfile
may be rerun with a partial command (followed by a carriage return).
nM> !stat<CR> nM> status myfile
Completion - Completing commands.
Commands may be completed by following the command name with a question mark (?). For example,
nM> status ? followed by a carriage return gives the prompt:
FILE=
and if the user enters 'myfile' followed by a carriage return, <CR> ,the command is completed as follows.
nM> STATUS,FILE=myfile
Intrinsics - shell control and other immediate commands
Primitives - data handling and processing commands
Macros - the NeXtMidas scripting language commands
UserMacros - Creating and Running user macros.
Creating and Running user macros.
User macros (or "%" macros) are typically created as a prototype before the developer has determined what the "signature" of the macro will be. User macros are not required to be "installed". That is, they do not have to be in the command dictionary. Any macro in the HOMEPATH area (see the HOMEPATH command) may be run with a prefix in front of the macro name. For example,
nM> homepath HomePath = /home/midas/nxm160/nextmidas/nxm/sys/mcr/
shows the home path is the MCR area of the SYS option tree. This means the demo command may be run in one of two ways: nM> demo OR nM> %demo
OS - Executing Operating System commands
How to run Operating System commands.
OS commands may be run in one of three ways:
- Internally
- nM> $os_command
- Externally
- nM> $$os_command
- Externally (Configured) - <os_command> MUST BE configured in an option
- tree with O-Support
- nM> os_command
Switches - See Users's Guide -> System Operations -> Switches
See the NeXtMidas Users's Guide -> System Operations -> Switches section for information on switches.
FILES - Working with data files of various types.
Working with data files of various types.
FILES - NeXtMidas file system support
The NeXtMidas environment was designed to provide a natural interace to any type of file on a system. The STATUS and DATALIST commands will work with any file supported on the system. File names are case sensitive, and should not have embedded white space.
The filename extension is used to determine the file group, where each group is handled by a different subclass of the BaseFile class. The file groups include Text, Image, Shape, Zip, Jar, and Midas Data files.
The underlying source of the data bytes is defined by the URL prefix. Each URL prefix is handled by a different subclass of the IOResource class. These include file:, http:, udp:, ram:, nfile:, stp:.
Auxes - How NeXtMidas finds files.
An AUX, or auxiliary, may be a directory or a URL. A collection of AUXES, the "read AUX list" is searched when looking for a file.
Auxes may be short strings or numbers in string form. Since X-Midas auxes must be numbers, many installations that use both will use numbers on the NeXtMidas side for consistency.
Example aux table:
WRITE - Aux name where new files are to be written READ - Comma seperated list of aux names to search
CWD - The current working directory HOME - Home path, where user (%) macros are kept. DAT - The SYS option tree DAT area. RAM - RAM memory 1 - User path (i.e. /user/midas/+/) 9 - ICE RamDisk 99 - The X-Midas install ../xm/dat/ area WEB - A WEB URL, configure this for your server
READ and WRITE are special reserved names that define the list of aux names currently in use.
Examples: TBD
Groups - Definition of Midas file groups.
File groups in NextMidas consist of:
- Data files (1000, 2000, ...)
- Image Files (.GIF, .TIF,...)
- Text Files (.XML, .TXT,...)
Resources - Definition of file resources
Types - Types of Midas Data files
There are a number of file types supported by NeXtMidas including: Type 1000, 2000, 3000, 5000, ...
Type1000 - One dimensional data Type2000 - Two dimensional data frames Type3000 - Record Oriented data structures Type4000 - Multiple Key=Value records Type5000 - Record Oriented with reference frame information Type6000 - Extended type3000 format
Type1000 - One dimensional data
This is typical for samples vs time data.
Type2000 - Two dimensional data frames
This is typical for samples vs time data.
Type3000 - Record Oriented data structures
This is typical for flat database files.
Type4000 - Multiple Key=Value records
This is typical for non-flat database files, or data with event keywords.
Type5000 - Record Oriented with reference frame information
This is typical for positions or orbit tracks to be plotted on maps.
Type6000 - Extended type3000 format
This is typical for flat database files.
TRIMMERS - Using trimmers to thin files in-line.
Trimmers of the form (offset1:offset2), where OFFSET1 is INCLUSIVE and OFFSET2 is EXCLUSIVE may be appended to a file name to indicate a subset of data within the file. Note that X-Midas used the form (start:end). This is a confusing construct with zero based accounting.
See User's Guide -> Files -> File Trimmers for more information and examples.
For example;
nM> noop datafile(3:10) outputfile
creates a file consisting of 7 elements starting at element 3 and including element 9. The THIN command uses the same principal of inclusive and exclusive offsets. See explain on THIN for more examples.
If the second arg starts with a "+", it is a relative distance to offset2, or the length of the segment.
If an argument starts with a "-", it is the offset relative to the end of the file. Thus file(-1:+1) refers to the last element of the file.
The trimmers can be registered in abscissa addressing by preceding the value with a tilde. By default, they are indexed. For instance:
nM> fft datafile(0:~10) outputfile
will FFT the first 10 seconds of the file, where
nM> fft datafile(~1:~10) outputfile
will FFT 10 seconds of the file starting 1 second into the file, where
If the arg to the left of the : is blank, the beginning of file is used. If the arg to the right of the : is blank, the end of file is used.
Trimmers must come before qualifiers. For example,
nM> noop datafile(0:nels){aux=cwd} outputfile
uses the AUX qualifier to indicate the file to be used is in the CWD (current working directory) AUX.
QUALIFIERS - Using qualifiers to modify file behavior.
Using qualifiers to modify file behavior.
QUALIFIERS - File qualifiers in NeXtMidas use curly braces, {}; trimmers use parentheses,
File qualifiers in NeXtMidas use curly braces, {}; trimmers use parentheses, (). Qualifiers are filename modifiers that change the normal interpretation of the file. They provide for specific value indexing, or other tuning parameters.
(N:M) - Trimmer, see help on trimmers
AUX - Aux specifier (overrides normal aux disk search paths) CFL - Circular File Length EXIST - URL existence check mask FS - Set the frame size in elements, 0=Type 1000 FG - Force file group (do not guess from extension) TAG - Set the ID (or tag) of a file FLAGS - Optional flags mask
When file qualifiers are masks, the also have the additional options: ALL - for all options (-1) NONE - none of the options (0) DEF - the default option (case dependent)
Tests can be combined by combining the strings with a '|' separator. For example: 'EXIST=CGI|STATUS' will apply CGI and STATUS tests for the exist qualifier.
File qualifiers are order independent although it is more readable to put file trimmers directly after the filename.
nM> noop fftdemo(N:M){AUX=DAT} outfile
Multiple file qualifiers may used together as follows:
nM> noop http://localhost:9000/fftdemo{EXISTS=NONE,AUX=DAT,FS=0} outfile{AUX=HOME,FLAGS=NATIVE}
AUX - This short-circuits the normal search through the read aux list.
This short-circuits the normal search through the read aux list. for the file only on the named aux.
nM> plot myfile{AUX=DAT}
TAG - This sets the ID of the file for plot layers.
This sets the ID of the file for plot layers.
nM> plot myfile{AUX=DAT,TAG=MYWORLD}
FLAGS - This sets optional flags to tailor the behavior of the file.
This sets optional flags to tailor the behavior of the file.
NATIVE - bypass Java I/O and use the OS's fopen,fclose,fread,fwrite calls. APPEND - append to the named file NOABORT - do not abort on error FLUSH - flush the header and data after every write.
nM> noop file1 file2{flags=NATIVE|APPEND}
FS - This qualifier can be used to override the frame size of a file on-the-fly.
This qualifier can be used to override the frame size of a file on-the-fly. FS>0 causes the input file to be treated as a type 2000 file with the specified frame size. It does not change the header of the file itself. Only the in-memory copy of the header is altered. If FS=0, the input file is cast as a type 1000 file.
FG - The FG (File Group) qualifier can be used to specify the category of the file.
The FG (File Group) qualifier can be used to specify the category of the file. This is useful if one wants to force a file to be treated as something other than what its extension might indicate. For instance, a perl script which generates a Midas data file may be looked as a test file by using this qualifier equal to TXT.
If there is no file type qualifier then NeXtMidas will attempt to determine the file type from the extension. Accepted values for the FG qualifier are:
DAT - Midas data file (Type 1000, 2000,...) IMG - Image file (.tif, .gif, .png,...) TXT - Text file
For example:
nM> datalist mytextfile.tmp{fg=txt}
opens mytextfile.tmp as a textfile even though it has a Midas DataFile extension.
EXIST - The EXIST qualifier is used to tailor the existence algorithm used for a
The EXIST qualifier is used to tailor the existence algorithm used for a particular URL. The values that are accepted and how they modify the algorithm are shown below.
NONE - Turn off any explicit existence checks. URLs are assumed to exist.
CGI - Performs a check on the URL string itself to determine if the URL
points to a CGI script. If the URL string contains known patterns then the existence is true.
LIST - An attempt is made to retrieve a directory listing of the directory
containing the URL. This listing is then examined to determine if the named file exists. In some contexts this is a very nice way to test for existence. The server and the directory must support server generated listings for this to work properly.
STATUS - A HEAD request is performed on the URL and the status code of the
response is examined. If the status code indicates success then the URL is assumed to exist. This has problems with CGI scripts since the script must be executed in order for the status code to be generated.
ALL - Use all of the tests.
TAG - The {TAG=name} qualifier may be used to set the "ID" of a file.
The {TAG=name} qualifier may be used to set the "ID" of a file. by the plot command to identify a layer.
CFL - The Circular File Length qualifier sets the length in seconds of the disk file
The Circular File Length qualifier sets the length in seconds of the disk file before wrapping back to the beginning. It behaves like a pipe but can be hours or days long.
TRIMMERS - Using trimmers to thin files in-line.
Using trimmers to thin files in-line.
Trimmers of the form (offset1:offset2), where OFFSET1 is INCLUSIVE and
OFFSET2 is EXCLUSIVE may be appended to a file name to indicate a subset
of data within the file. Note that X-Midas used the form (start:end).
This is a confusing construct with zero based accounting.
See User's Guide -> Files -> File Trimmers for more information and examples.
For example;
nM> noop datafile(3:10) outputfile
creates a file consisting of 7 elements starting at element 3 and including element 9. The THIN command uses the same principal of inclusive and exclusive offsets. See explain on THIN for more examples.
If the second arg starts with a "+", it is a relative distance to offset2, or the length of the segment.
If an argument starts with a "-", it is the offset relative to the end of the file. Thus file(-1:+1) refers to the last element of the file.
The trimmers can be registered in abscissa addressing by preceding the value with a tilde. By default, they are indexed. For instance:
nM> fft datafile(0:~10) outputfile
will FFT the first 10 seconds of the file, where
nM> fft datafile(~1:~10) outputfile
will FFT 10 seconds of the file starting 1 second into the file, where
If the arg to the left of the : is blank, the beginning of file is used. If the arg to the right of the : is blank, the end of file is used.
Trimmers must come before qualifiers. For example,
nM> noop datafile(0:nels){aux=cwd} outputfile
uses the AUX qualifier to indicate the file to be used is in the CWD (current working directory) AUX.
HELP - Help with this HELP engine
Help with this HELP engine
This facility covers the HELP system.
Commands - TREE - Show the help tree of topics/sections.
TREE - Show the help tree of topics/sections.
Usage - This section describes how to use help
This section describes how to use help
Building_Help_Files - Help files are built by writing text as it should appear in a help browser.
Help files are built by writing text as it should appear in a help browser. NextMidas supports a markup language that is a minor extension of the X-Midas markup language. In this language, most keywords are indicated by a tilde (~) as the first character in a line excluding white space. One exception is the See Also keyword which can appear anywhere in the text without a tilde.
form: ~Main text
The text here shows up in the main help screen for this file as is. This allows comments to be inserted among the section bullets to categorize or comment on different sections.
form: ~Comment <comment text>
The Comment keyword will cause the remaining text of the line to be completely ignored. Comments cannot span multiple lines so each line of a multiline comment must be prefixed with ~Comment.
form: ~Include <fname>
Causes the text of the specified filename to be read completely as if the file contents where present here. Analogous to the C preprocessor include directive.
form: ~Section <section name> text
When a Section keyword is found, the next token is taken to be the section name. Everything from the next line to the next Section, File or End token is taken to be the contents of the token. Sections cannot be nested.
<subsection name>
text
When a SubSection keyword is found, the next token is taken to be the subsection name. Everything from the next line to the next SubSection, Section, File or End token is taken to be the contents of the token. SubSections cannot be nested. They can be addressed in See Also references by using a section#subsection syntax.
or - closes on the previous section.
closes on the previous section.
form: SEE ALSO: <opt> <area> <name>
The See Also keyword is processed up to the next period or end of line. It cannot span multiple lines. Multiple items can be comma separated. If <opt> or <area> are not specified, they are defaulted from the current page or previous entry in a comma separated list.
Currently supported areas are HLP, EXP, and DOC.
NOTE: The ALSO and the colon are optional.
NotFound - This help section entry was not found.
There is a malformed help file or the path was restricted while performing HELP/BUILD.
PIPES - Memory files, writing "piped" macros.
Memory files, writing "piped" macros.
To increase the default pipe size use /PS=<size> on your macro or use the {PS=<size>} qualifier to change the pipe size on individual pipes.
SHELL - Using the NeXtMidas Shell, command syntax.
Using the NeXtMidas Shell, command syntax.
Use $ or $$ to run an OS command from the shell. nM> $$cd /home/midas
"One-shot" commands may be run with: nm command_name param1 ... paramn
Switches - See Users's Guide -> System Operations -> Switches
See the NeXtMidas Users's Guide -> System Operations -> Switches section for information on switches.
RMIF - Remote Midas InterFace Protocol
Remote Midas InterFace Protocol
RMIF - the Remote Midas InterFace protocol.
The RMIF protocol is a take-off on Java's RMI (Remote Method Invocation).
Java's RMI is designed to make objects at a remote site accessible as if they are local objects. A local reference to the remote object is created and method calls on the local reference are passed to the remote object via a transport layer, usually TCPIP. The method name and its arguments are serialized, sent to the remote processor, and executed. The optional resulting object is serialized, sent back to the local processor and reconstructed in the objects local reference.
The main purpose of the RMIF protocol is to allow NeXtMidas GUIs to control Midas applications at remote sites. While the protocol may be applied to other problems, it's primary goal is to minimize link bandwidth for data displays and respond robustly to network drop-outs common to real-world long-distance multi-hop links.
RMI - Java's Remote Method Invocation protocol.
RMI was not used directly for a number of reasons:
- the server side applications are not typically Java based (C,C++,FORTRAN).
- the standard RMI implementation is layered on TCP, which severely
- limits throughput and robustness over real-world links.
- object serialization requires unnecessary over-head and
- data conversions to/from network ordering for data displays.
The RMI reference was kept to make it easier for developers that may have experience with Java RMI. The concepts and methodology were paralleled wherever possible.
The current implementation of RMIF is layered on top of UDP. All routing to and from remote objects is handled by the UDP header. There is no subrouting within the RMIF protocol. An RMIF object is defined by a node name (or network address) and a port number.
Header - The RMIF 8-byte packet header.
The UDP payload begins with an 8-byte RMIF header. The RMIF protocol uses the 8-byte header to, in effect, compress the overhead that would be necessary to serialize a method call. The header bytes (in order) are:
FUNC - defines the method to call FLAG - is the protocol version number and options mask INFO - channel number for fast routing of monitored properties REP - packet data representation (VAX|IEEE|EEEI) SEQ - sequence number (0-127) for Reliable packets TRY - number of re-sends for Reliable packets RPT - report sequence number ADJ - number of bytes in adjunct header
The size of the data portion of the payload is the UDP packet size minus the 8 byte header and the adjunct header if ADJ>0.
AdjunctHeader - The RMIF adjunct header.
The optional adjunct header follows directly after the 8-byte RMIF header. It must be a multiple of 8-bytes as well for alignment purposes. The currently supported adjunct header options are:
ADJ_TIME - an 8-byte double precision time in J1950 seconds ADJ_OFFSET - an 8-byte double precision file offset for MFTP
In each of these cases, the ADJ field in the RMIF header equals 8. The data portion of the payload will start at the 16th byte instead of the 8th.
Functions - The currently defined RMIF functions.
The currently supported functions (or method calls) are:
GET - request the value of a named property RET - return the value for a GET request SET - set a named property ACK - acknowledge the value for a SET request OPEN - open connection to a remote or a channel to one of it's properties OPENED - acknowledges an OPEN request CLOSE - close connection to a remote or a channel to one of it's properties CLOSED - acknowledges an CLOSE request MODIFY - modify a channel connection PING - request keep-alive status PONG - return keep-alive status DHDR - process data header DBUF - process data buffer DBUFC - process compressed data buffer DBUFD - process compressed data buffer GETPROP - request list of published properties RETPROP - return number of properties or property names RECEIPT - acknowledge receipt of a reliable packet MFTP - process the Midas File Transfer Protocol
Methods - How RMIF packets are converted to function calls.
The RMIish components of the protocol are accomplished with the GET, RET, SET, and ACK functions. The Java language suggests the use of public methods beginning with "set" and "get" to provide a common interface to an object's public properties. To allow combining multiple SETs or GETs in a single network packet, the data portion of the GET|RET|SET|ACK packet is a serialized table class. The key for each table entry is the body of the method name, and the value for each key is the method's argument. For example, to call the setFreq, setAmp, and setOptions methods on a remote object the data portion of a RMIF packet with the FUNC=SET would be:
{FREQ=1.3E6,AMP=1,OPTIONS={MODE=1,AXIS=NONE}}
The syntax is all ASCII to allow using off-the-shelf tools or XML parsers to monitoring network traffic. Note that the setOptions() method has more than one argument. To do this, the arguments are expressed as another table, with each argument being named. This again aids in monitoring/debugging.
RMIvsRMIF - Why RMIF and UDP instead of RMI and TCP.
The RMIF protocol uses considerably less bandwidth than RMIs serialization of calls for most functions. Also, TCP socket writes eventually end up in TCP packets that have more overhead than UDP. In addition, when and where socket writes get broken into TCP packets is not under the application's control. If network dropouts cause a TCP packet to be dropped, this may or may not lie on a functional boundary for a method call making graceful recovery difficult.
Another benefit of UDP is throughput. In TCP, system drivers usually limit the number of outstanding packets to 12. This means that until the remote connection receives the packet and acknowledges receipt, the local socket will hold-off outgoing packets. Using the typical maximum transfer unit, MTU, of 1500 bytes and roundtrip delay of 500 ms, we can calculate a maximum sustained transfer rate of 12*1500/.5 = 36kby/sec.
The RMIF protocol implements the reliable portion of the TCP layer at the user level, allowing the application to set the maximum number of outstanding packets (default=12,max=128) and which packets to send reliably or unreliably. Typically, data display packets are sent unreliably, and all others reliably. Unreliable packets set the SEQ header byte to -1. Reliable packets have SEQ set to a one-up counter (0-127) by the sender, along with a retry count TRY. If the sender does not get a receipt after a tunable tolerance plus the last measured round trip delay, it will resend the packet. By combining the reliable and unreliable streams in the same socket, we can implement priority schemes that, for instance, favor controls over data.
Another benefit of UDP is that it is connectionless. A UDP port on a machine is basically a bucket on a node to catch network packets that are thrown at it. They can be thrown from anywhere, from any number of sources, in any order. Each packet contains the network address and port number of who threw the packet, so you can throw something back if called for. Then you're done. No opens, no closes. Network problems cannot cause connections to hang in an unknown state, because there never is a connection. This leaves the timing out and recovery to the application, where appropriate tuning may be applied.
Most of the protocol handling will be administered by an RMIF primitive or component on each end of the connection. This primitive should insulate the framework from the RMIF protocol and convert the method calls to appropriate constructs for that framework. See the RMIF Explain File for more details.
MFTP - The Midas File Transfer Protocol
An RMIF connection also supports file transfers in both directions. The client typically requests a file from the server, however, the client can post a file to the server if allowed.
The low level packets for accessing a file are as follows:
Client: func=MFTP data={FUNC=EXIST,NAME=filename} Server: func=MFTP data={FUNC=EXIST,NAME=filename,EXISTS=Y|N}
Client: func=MFTP data={FUNC=OPEN,NAME=filename,RATE=byte/sec,PKTLEN=bytes} Server: func=MFTP data={FUNC=OPEN,NAME=filename,LENGTH=bytesAtOpen}
Client: func=MFTP data={FUNC=READ,NAME=filename,OFFSET=byte,LENGTH=bytes,RATE=byte/sec,PKTLEN=bytes} Server: func=MFTP data={FUNC=READ,NAME=filename,OFFSET=byte,LENGTH=availbytes} Server: func=DBUF adj=offset data=binaryDataBuffer (multiple packets)
Client: func=MFTP data={FUNC=CLOSE,NAME=filename} Server: func=MFTP data={FUNC=CLOSE,NAME=filename,LENGTH=bytesAtClose}
The RATE and PKTLEN specifiers are optional. They default to RATE=2e6 and PKTLEN=8K.
This level of protocol deals with raw files or an IOResource in NeXtMidas. It does not understand Midas file headers, detached headers, or detached packet files. This must be handled by the calling routine, usually the DataFile class.
For example, an RMIF server on port 9001 can serve a file using the command:
nM> noop/gpw mftp://remoteHostName:9001/remoteFileName localFileName
where the /gpw invokes a graphical progress widget.