Current File : //usr/man/man1/ncftpspooler.1

'\" te
.TH ncftpspooler 1 "NcFTP Software" ncftpspooler
.SH NAME
ncftpspooler - Global batch FTP job processor daemon
.SH "SYNOPSIS"
.PP
.B ncftpspooler
\-d
.RI [ "options" ]
.PP
.B ncftpspooler
\-l
.RI [ "options" ]
.\"-------
.SH "OPTIONS"
.\"-------
.SS
Command line flags:
.TP 8
.B "-d"
Begin background processing of FTP jobs in the designated
FTP job queue directory.
.TP 8
.BI \-q " XX"
Use this option to specify a directory to use as the FTP job queue
instead of the default directory, /var/spool/ncftp.
.TP 8
.BI \-o " XX"
Use this option to specify a filename to use as the log file.
By default,
(and rather inappropriately) the program simply uses a file called
.I log
in the job queue directory.
If you don't want a log, use this option to specify /dev/null.
.TP 8
.B \-l
Lists the contents of the job queue directory.
.TP 8
.BI \-s " XX"
When the job queue is empty, the program sleeps 120 seconds and then
checks again to see if a new job has been submitted.
Use this option to change the number of seconds used for this delay.
.\"-------
.SH "DESCRIPTION"
.\"-------
.PP
The
.I ncftpspooler
program evolved from the
.I ncftpbatch
program.
The
.I ncftpbatch
program was originally designed as a ``personal FTP spooler'' which
would process a single background job a particular user and exit when
it finished;
the
.I ncftpspooler
program is a ``global FTP spooler'' which stays running and 
processes background jobs as they are submitted.
.PP
The job queue directory is monitored for specially-named and formatted
text files.
Each file serves as a single FTP job.
The name of the job file contains the type of FTP job
.RI ( get " or " put "),"
a timestamp indicating the earliest the
job should be processed,
and optionally some additional information to make it easier to create
unique job files
(i.e. a sequence number).
The contents of the job files have information such as the remote
server machine to FTP to, username, password, remote pathname, etc.
.PP
Your job queue directory must be readable and writable by the user
that you plan to run
.I ncftpspooler
as, so that jobs can be removed or renamed within the queue.
.PP
More importantly, the user that is running the program will need
adequate privileges to access the local files that are involved in
the FTPing.
I.e., if your spooler is going to be processing jobs which upload files
to remote servers, then the user will need read permission on the local files
that will be uploaded (and directory access permission the parent directories).
Likewise, if your spooler is going to be processing jobs which download
files, then the user would need to be able to write to the local
directories.
.PP
Once you have created your spool directory with appropriate
permissions and ownerships, you can run
.I "ncftpspooler\ \-d"
to launch the spooler daemon.
You can run additional spoolers if you want to process more than FTP
job from the same job queue directory simultaneously.
You can then monitor the log file
.RI "(i.e., using " "tail\ \-f" " )"
to track the progress of the spooler.
Most of the time it won't be doing anything, unless job files
have appeared in the job queue directory.
.\"-------
.SH "JOB FILE NAMES"
.\"-------
.PP
When the
.I ncftpspooler
program monitors the job queue directory, it ignores any files that do
not follow the naming convention for job files.
The job files must be prefixed in the format of
.I "X\-YYYYMMDD\-hhmmss"
where
.I X
denotes a job type,
.I YYYY
is the four-digit year,
.I MM
is the two-digit month number,
.I DD
is the two-digit day of the month,
.I hh
is the two-digit hour of the day (00\-23),
.I mm
is the two-digit minute,
and
.I ss
is the two-digit second.
The date and time represent the earliest time you want the job to be run.
.PP
The job type can be
.I g
for a get (download from remote host),
or
.I p
for  aput (upload to remote host).
.PP
As an example, if you wanted to schedule an upload to occur at 11:45 PM
on December 7, 2001, a job file could be named
.RS 4
.sp
.br
p-20011207-234500
.br
.RE
.PP
In practice, the job files include additional information such as a
sequence number or process ID.
This makes it easier to create unique job file names.
Here is the same example, with a process ID and a sequence number:
.RS 4
.sp
.br
p-20011207-234500-1234-2
.br
.RE
.PP
When submitting job files to the queue directory, be sure to use
a dash character after the
.I hhmmss
field if you choose to append any additional data to the job file name.
.\"-------
.SH "JOB FILE CONTENTS"
.\"-------
.PP
Job files are ordinary text files, so that they can be created by
hand.
Each line of the file is a key-pair in the format
.IR "variable" "=" "value" ","
or is a comment line beginning with an
octothorpe character
.RI ( "#" "),"
or is a blank line.
Here is an example job file:
.RS 4
.sp
.br
# This is a NcFTP spool file entry.
.br
job-name=g-20011016-100656-008299-1
.br
op=get
.br
hostname=ftp.freebsd.org
.br
xtype=I
.br
passive=1
.br
remote-dir=pub/FreeBSD
.br
local-dir=/tmp
.br
remote-file=README.TXT
.br
local-file=readme.txt
.br
.RE
.PP
Job files are flexible since they follow an easy-to-use format and do not
have many requirements, but there are a few mandatory parameters that must
appear for the spooler to be able to process the job.
.TP 8
.I op
The operation (job type) to perform.
Valid values are
.IR "get" " and " "put" "."
.TP 8
.I hostname
The remote host to FTP to.
This may be an IP address or a DNS name (i.e.
ftp\.example\.com).
.PP
For a regular
.I get
job, these parameters are required:
.TP 8
.I remote\-file
The pathname of the file to download from the remote server.
.TP 8
.I local\-file
The pathname to use on the local server for the downloaded file.
.PP
For a regular
.I put
job, these parameters are required:
.TP 8
.I local\-file
The pathname of the file to upload to the remote server.
.TP 8
.I remote\-file
The pathname to use on the remote server for the uploaded file.
.PP
For a recursive
.I get
job, these parameters are required:
.TP 8
.I remote\-file
The pathname of the file or directory to download from the remote server.
.TP 8
.I local\-dir
The directory pathname to use on the local server to contain the downloaded items.
.PP
For a recursive
.I put
job, these parameters are required:
.TP 8
.I local\-file
The pathname of the file or directory to upload to the remote server.
.TP 8
.I remote\-dir
The directory pathname to use on the remote server to contain the uploaded items.
.PP
The rest of the parameters are optional.
The spooler will attempt to use reasonable defaults for these
parameters if necessary.
.TP 8
.I user
The username to use to login to the remote server.
Defaults to ``anonymous'' for guest access.
.TP 8
.I pass
The password to use in conjunction with the username to login to the remote server.
.TP 8
.I acct
The account to use in conjunction with the username to login to the remote server.
The need to specify this parameter is extremely rare.
.TP 8
.I port
The port number to use in conjunction with the remote hostname to connect
to the remote server.
Defaults to the standard FTP port number, 21.
.TP 8
.I host-ip
The IP address to use in conjunction with the remote hostname to connect
to the remote server.
This parameter can be used in place of the
.I hostname
parameter, but one or the other must be used.
This parameter is commonly included along with the
.I hostname
parameter as supplemental information.
.TP 8
.I xtype
The transfer type to use.
Defaults to binary transfer type (TYPE\ I).
Valid values are
.I I
for binary,
.I A
for ASCII text.
.TP 8
.I passive
Whether to use FTP passive data connections (PASV) or FTP active
data connections (PORT).
Valid values are
.I 0
for active,
.I 1
for passive,
or
.I 2
to try passive, then fallback to active.
The default is
.IR 2 "."
.TP 8
.I recursive
This can be used to transfer entire directory trees.
By default, only a single file is transferred.
Valid values are
.I yes
or
.IR no "."
.TP 8
.I delete
This can be used to delete the source file on the source machine
after successfully transferring the file to the destination machine.
By default, source files are not deleted.
Valid values are
.I yes
or
.IR no "."
.TP 8
.I job\-name
This isn't used by the program, but can be used by an entity which
is automatically generating job files.
As an example, when using the
.I \-bbb
flag with
.IR ncftpput ,
it creates a job file on stdout with a
.I job\-name
parameter so you can easily copy the file to the
job queue directory with the suggested job name as the job file name.
.TP 8
.I pre\-ftp\-command
.TP 8
.I post\-ftp\-command
These parameters correspond to the
.IR "\-W" ", and " "\-Y"
options of
.I ncftpget
and
.IR ncftpput "."
It is important to note that these refer to RFC959 File Transfer Protocol
commands and
.B not
shell commands, nor commands used from within /usr/bin/ftp or ncftp.
.TP 8
.I pre\-shell\-command
.TP 8
.I post\-shell\-command
These parameters provide hooks so you can run a custom program when
an item is processed by the spooler.
Valid values are pathnames to scripts or executable programs.
Note that the value must not contain any command-line arguments -- if
you want to do that, create a shell script and have it run your program
with the command-line arguments it requires.
.PP
Generally speaking, 
.I post\-shell\-command
is much more useful than
.I pre\-shell\-command
since if you need to use these options you're more likely to want to
do something after the FTP transfer has completed rather than before.
For example, you might want to run a shell script which pages
an administrator to notify her that her 37 gigabyte file download has completed.
.PP
When your custom program is run, it receives on standard input the 
contents of the job file (i.e. several lines of
.IR variable = value
key-pairs), as well as additional data the spooler may provide, such
as a
.I result
key-pair with a textual description of the job's completion status.
.PP Here is a sample Perl script which demonstrates having a
.I post\-shell\-command
update a log file named
/var/log/ncftp_spooler.
.RS 4
.sp
.br
#!/usr/bin/perl -w
.br
.sp
my ($line);
.br
my (%params) = ();
.br
.sp
while (defined($line = <STDIN>)) {
.br
	$params{$1}\ =\ $2
.br
		if\ ($line\ =~\ /^([^=\\#\\s]+)=(.*)/);
.br
}
.br
.sp
if ((defined($params{"result"})) &&
.br
  ($params{"result"} =~ /^Succeeded/))
.br
{
.br
	open(LOG, ">> /var/log/ncftp_spooler.log")
.br
		or exit(1);
.br
	print LOG "DOWNLOAD" if ($params{"op"} eq "get");
.br
	print LOG "UPLOAD" if ($params{"op"} eq "put");
.br
	print LOG " ", $params{"local-file"}, "\\n";
.br
	close(LOG);
.br
}
.br
.RE
.\"-------
.SH "DIAGNOSTICS"
.\"-------
.PP
The log file should be examined to determine if any
.I ncftpspooler
processes are actively working on jobs.
The log contains copious amounts of useful information, including
the entire FTP control connection conversation between the FTP
client and server.
.\"-------
.SH "BUGS"
.\"-------
.PP
The
.I recursive
option may not be reliable since 
.I ncftpspooler
depends on functionality which may or may not be present in the
remote server software.
Additionally, even if the functionality is available, 
.I ncftpspooler
may need to use heuristics which cannot be considered 100% accurate.
Therefore it is best to create individual jobs for each file in the
directory tree, rather than a single recursive directory job.
.PP
For resumption of downloads to work, the remote server must support
the FTP
.I SIZE
and
.I MDTM
primitives. 
Most modern FTP server software can do this, but there are still a number
of bare-bones
.I ftpd
implementations which do not.
In these cases,
.I ncftpspooler
will re-download the file in entirety each time until the download succeeds.
.PP
The program needs to be improved to detect jobs that have no chance of
ever completing successfully.
There are still a number of cases where jobs can get spooled but get
retried over and over again until a vigilant sysadmin manually removes
the jobs.
.PP
The spool files may contain usernames and passwords stored in cleartext.
These files should not be readable by any user except the user running the
program!
.\"-------
.SH "AUTHOR"
.\"-------
.PP
Mike Gleason, NcFTP Software (http://www.ncftp.com).
.\"-------

.\" Oracle has added the ARC stability level to this manual page
.\" on April 20, 2011
.SH ATTRIBUTES
See
.BR attributes (5)
for descriptions of the following attributes:
.sp
.TS
box;
cbp-1 | cbp-1
l | l .
ATTRIBUTE TYPE	ATTRIBUTE VALUE 
=
Availability	network/ftp/ncftp
=
Stability	Volatile
.TE 
.PP
.SH "SEE ALSO"
.\"-------
.PP
.IR ncftpbatch(1),
.IR ncftp (1),
.IR ncftpput (1),
.IR ncftpget (1),
.IR uucp (1).


.SH NOTES

.\" Oracle has added source availability information to this manual page
.\" on April 20, 2011
This software was built from source available at https://java.net/projects/solaris-userland.  The original community source was downloaded from  ftp://ftp.ncftp.com/ncftp/older_versions/ncftp-3.2.3-src.tar.bz2

Further information about this software can be found on the open source community website at http://www.ncftp.com/ncftp/.