[[FXP.One:Clients|FTP client]]. It is in fact a very flexible FTP/FXP engine. This engine does
all the hard work with dealing with FTP sites. Built into that is
a very simple but powerful API protocol. The idea is then, if someone
wants to do an FTP, or FXP, client they can then make one without
the FTP hassles. Currently there already are multiple [[FXP.One:Clients|clients]]. They
all talk to the FXP.One engine, and you can interchange the [[FXP.One:Clients|clients]].
Define where to connect for the FXP.One engine. It can be the same server as clomps run on, use localhost. If you leave the '''PASS=''' out, then clomps will prompt for passwords.
That is, use one client to create and queue up some items and start the
'''ssl=forced''' on this line refers to the connection to the engine. Encryption is a good idea, except if you are using localhost, and nobody else has root on your server.
queue process. At a later time, a different client, from a different location
can connect and check on the progress of that queue, change it, add to it and
Here we define the sites to connect to, and which directories to list. Note that '''home''' has two directories listed, they are combined.
You do not need to specify '''LAST_CHECK''', it is added by clomps automatically. Leave it out if you are creating a new conf. It has the "newest most item's date" stored, so we know what is newer, for each site.
If you run this clomps.conf now, you will get an output like:
'''FXP.One V2 released! Now comes with HTML5 WebSocket FTP Client! IRC Auto-trader bot! '''
./clomps -vf example.conf
The FXP.One engine features:
* Full FTP and FXP capabilities.
connected to FXP.One
* SSL/TLS support, auto-sensing and forced.
successfully negotiated SSL
* SSL/TLS data support, auto-sensing and forced.
Successfully connected to FXP.One
* SSCN secure data FXP support.
Site glftpdSite processing dir '/tv-x264/'
* CCSN secure data FXP support.
Site home processing dir '/tv/'
* XDUPE aware for faster queue processing.
Site Another processing dir '/tv-x264/'
* Auto resume, or overwrite options
Site ASite1 processing dir '/x264-tv/'
* Resume last (re-queue all resume items last for faster queue processing)
Site home processing dir '/drive2/tv/'
* FXP direction control (if one site is firewalled)
Another finished processing: 111 files ( 1/ 4)
* PRET Pre-transfer support for ring-sites.
home finished processing: 3135 files ( 2/ 4)
* Skip lists for both files and directories
glftpdSite finished processing: 158 files ( 3/ 4)
* Pass lists for both files and directories (opposite to skip list)
ASite1 finished processing: 166 files ( 4/ 4)
* Move-first for both files and directories
* Automatic skip of empty files and directories.
* Encodes all file and directory names as to handle any locale.
-----======> DISPLAY MATRIX <======-----
T |ASite|Anoth|glftp|home | New Items
--+-----+-----+-----+-----+-----------
d |INC |MISS |MISS |MISS | Best.Laid.Plans.2012.1080p.BluRay.X264-7SinS
d |INC |MISS |MISS |MISS | To.Catch.A.Thief.1955.1080p.BluRay.x264-HD4U
d | |MISS |MISS |MISS | The.Summer.Of.Massacre.2011.720p.BluRay.x264-UNTOUCH
ABLES
d | |MISS |NUKE |MISS | Under.Still.Waters.2008.720p.BluRay.x264-SWAGGERHD
Which shows new releases, by showing up (names on the right side) they exist somewhere. Any site column with a blank entry, means the release is there and complete. '''INC''' means that the release is incomplete (INCTEST patten matches), and similarly '''NUKE''' means nuked. '''MISS''' means this release is not on this site.
=== Automatic queueing ===
In addition to finding new entries, you can ask clomps to create queues. The queue lines are defines with '''pass number'''. Starting from 1. This means, all queue lines with pass=1 will be started at the same time. Once all pass=1 transfers are complete, clomps will run pass=2. This allows for all-concurrent queueing, or, sequentially queueing.
[[Image:ufxp.screen1.jpg|right|thumb|100px|UFxp]]
For example (Spaces added between '|' separator for readability):
Which would automatically transfer items from '''glftpdSite''' "/tv-x264/", to '''ASite1''' "/x264-tv/". But only releases matching "the.simpson*" etc, unless they are *FINSUB* etc. The '''INCSKIP''' tells clomps not to transfer things that are incomplete. Clomps will then only transfer completed-releases. If you remove '''INCSKIP''' clomps will start to move incomplete releases as well.
The right-hand side in ACCEPT and REJECT statements are slash-separated list of matches. Ie, "*simpsons*/blue.planet*" would match on "*simpsons*" and "blue.planet*", with glob-style pattern matching. This means "I.hate.simpsons" would match (in the middle) but "the.blue.planet" would not (since it was not "*blue.planet*". No starting "*"). Note that this would also fail on "Blue_Planet" and "blue planet", since the period "." is literal. The best way is to use "blue?planet*". All matches are '''case-insensitive'''.
After pass=1 has completed, move '''everything''' from '''ASite1''' to home. Except for dvdrips. Incompletes will also be started.
[[Compile]] FXP.One
=== Clomps-irc ===
[[FXP.Oned_Changelog|Changelog]]
A separate binary is compiled for the IRC Bot version of clomps. Clomps-irc connects to an IRC server(s), and sits in specified channel(s) to look for "New Release" notifications (or "Completion" messages depending on style).
[http://www.lundman.net/unix/FXP.One_commitlog.txt Old CVS Changelog]
To define an ircbot to listen to an irc server / irc channel, add the following line:
[[Encoding]] The directory and file encoding scheme used.
'''NICK=''' Is a Regular Expression (RE) match on the nick name. Clomps will ignore other nicks. Here "(?i)" is the PCRE for "case-insensitive", matching on nick "thebot".
[[Site Definition]] Current SITE definition and its members.
'''MATCH''' is Regular Expression to match a release on. This does;
* Match only on section "TV" (Since RE started with "[TV]")
[[Extra Site]] members also understood by the engine.
* Match on new release message ("New Release", which could also be "Completed" or similar messages), to differentiate to "New racer, New SFV, or Halfway-through announces).
* Match and parse out the release name, ie "(.*)" and place it in "${releasename}". The part that is in parenthesis. This would unfortunately have to be tuned for each ircbot's output.
* If "${releasename}" passes the ACCEPT and REJECT rules, in this case "*simpsons* and *FINSUB*;
The release is then moved from SRCSITE:/SRCDIR/${releasename} -> DSTSITE:/DSTDIR/${releasename}
[[Client Site]] recommendations.
Should we add SESSIONS=<int> as well? So clomps-irc can be told to limit the number of sessions? Some cases you only ever want to use SESSIONS=1, so all new releases are added to the end of currently active transfers.
[[Phases]] required to do a file transfer. (need to do directory listing some day)
If we allow SESSIONS=2 or more, should we add balancing? round-robin? least-entries-in-queue?
==API COMMANDS==
[[Protocol]] definition and specifications.
== Local transfers ==
[[FXPOne:local|local]]
== Help ==
[[FXPOne|FAQ]]
Revision as of 07:07, 29 February 2012
FXP.One is a whole new system to FTP and FXP. It is not just another
FTP client. It is in fact a very flexible FTP/FXP engine. This engine does
all the hard work with dealing with FTP sites. Built into that is
a very simple but powerful API protocol. The idea is then, if someone
wants to do an FTP, or FXP, client they can then make one without
the FTP hassles. Currently there already are multiple clients. They
all talk to the FXP.One engine, and you can interchange the clients.
That is, use one client to create and queue up some items and start the
queue process. At a later time, a different client, from a different location
can connect and check on the progress of that queue, change it, add to it and
so on.
<paypal></paypal>
FXP.One V2 released! Now comes with HTML5 WebSocket FTP Client! IRC Auto-trader bot!
The FXP.One engine features:
* Full FTP and FXP capabilities.
* SSL/TLS support, auto-sensing and forced.
* SSL/TLS data support, auto-sensing and forced.
* SSCN secure data FXP support.
* CCSN secure data FXP support.
* XDUPE aware for faster queue processing.
* Auto resume, or overwrite options
* Resume last (re-queue all resume items last for faster queue processing)
* FXP direction control (if one site is firewalled)
* PRET Pre-transfer support for ring-sites.
* Skip lists for both files and directories
* Pass lists for both files and directories (opposite to skip list)
* Move-first for both files and directories
* Automatic skip of empty files and directories.
* Encodes all file and directory names as to handle any locale.
