Clomps conf: Difference between revisions

From Lundman Wiki
No edit summary
mNo edit summary
Line 1: Line 1:
==The FXP.One clients==
# Which FXP.One to connect to. Omit PASS to have app prompt for passwd.
FXPONE|HOST=server.that.runs.fxpone.com|PORT=8885|USER=admin|PASS=admin|ssl=forced


This is a known list of [[FXP.One]] clients. If you make a new one, feel free to add it here.  
Define where to connect for the FXP.One engine. It can be the same server as clomps run on, use localhost. If you leave the '''PASS=''' out, then clomps will prompt for passwords.
'''ssl=forced''' on this line refers to the connection to the engine. Encryption is a good idea, except if you are using localhost, and nobody else has root on your server.
# Define sites and paths.
SITE|NAME=ASite1|DIR=/x264-tv/|USESKIP=1|INCTEST=%s-(INCOMPLETE)|NUKETEST=NUKED-%s|LAST_CHECK=1329887220
SITE|NAME=Another|DIR=/tv-x264/|USESKIP=1|INCTEST=%s-(INCOMPLETE)|NUKETEST=NUKED-%s|LAST_CHECK=1329897720
SITE|NAME=glftpdSite|DIR=/tv-x264/|USESKIP=1|INCTEST=(INCOMPLETE)-%s|NUKETEST=!NUKED-%s|LAST_CHECK=1329919260
SITE|NAME=home|DIR=/tv/|DIR=/drive2/tv/|USESKIP=1|INCTEST=%s-(INCOMPLETE)|NUKETEST=NUKED-%s|LAST_CHECK=1329828420


Here we define the sites to connect to, and which directories to list. Note that '''home''' has two directories listed, they are combined.
You do not need to specify '''LAST_CHECK''', it is added by clomps automatically. Leave it out if you are creating a new conf. It has the "newest most item's date" stored, so we know what is newer, for each site.


=== FXP.cOne ===
If you run this clomps.conf now, you will get an output like:


'''* FXP.cOne (CDK/ncurses client)'''
./clomps -vf example.conf
connected to FXP.One
successfully negotiated SSL
Successfully connected to FXP.One
Site glftpdSite processing dir '/tv-x264/'
Site home processing dir '/tv/'
Site Another processing dir '/tv-x264/'
Site ASite1 processing dir '/x264-tv/'
Site home processing dir '/drive2/tv/'
      Another finished processing:  111 files (  1/  4)
          home finished processing: 3135 files (  2/  4)
    glftpdSite finished processing:  158 files (  3/  4)
        ASite1 finished processing:  166 files (  4/  4)
-----======> DISPLAY MATRIX <======-----
T |ASite|Anoth|glftp|home | New Items
--+-----+-----+-----+-----+-----------
d |INC  |MISS |MISS |MISS | Best.Laid.Plans.2012.1080p.BluRay.X264-7SinS
d |INC  |MISS |MISS |MISS | To.Catch.A.Thief.1955.1080p.BluRay.x264-HD4U
d |    |MISS |MISS |MISS | The.Summer.Of.Massacre.2011.720p.BluRay.x264-UNTOUCH
ABLES
d |    |MISS |NUKE |MISS | Under.Still.Waters.2008.720p.BluRay.x264-SWAGGERHD
Which shows new releases, by showing up (names on the right side) they exist somewhere. Any site column with a blank entry, means the release is there and complete. '''INC''' means that the release is incomplete (INCTEST patten matches), and similarly '''NUKE''' means nuked. '''MISS''' means this release is not on this site.


[[Image:fxp.cone.screen1.jpg|right|thumb|100px|FXP.cOne]]


This [[FXP.One]] client ships with the FXP.One packages and will be compiled automatically if '''./configure''' can find '''ncurses''' and the special version of '''CDK'''. Download the CDK library from [http://invisible-island.net/cdk/ http://invisible-island.net/cdk/]
=== Automatic queueing ===


Until the local site can be added the client, if you wish to create a '''local''' site, please use ''<local>'' as the setting for '''host''', '''user''' and '''pass'''.
In addition to finding new entries, you can ask clomps to create queues. The queue lines are defines with '''pass number'''. Starting from 1. This means, all queue lines with pass=1 will be started at the same time. Once all pass=1 transfers are complete, clomps will run pass=2. This allows for all-concurrent queueing, or, sequentially queueing.  


For example (Spaces added between '|' separator for readability):


Status: Beta: Completely Functional, but missing much cosmetics. Knowns bugs exist.
AUTOQ|PASSNUM=1|FROM=glftpdSite|TO=ASite1 | ACCEPT=the.simpson*/family.guy*/american.dad*/How?I?met?your?mother* | REJECT=*FINSUB*/*AC3* | INCSKIP


Which would automatically transfer items from '''glftpdSite''' "/tv-x264/", to '''ASite1''' "/x264-tv/". But only releases matching "the.simpson*" etc, unless they are *FINSUB* etc. The '''INCSKIP''' tells clomps not to transfer things that are incomplete. Clomps will then only transfer completed-releases. If you remove '''INCSKIP''' clomps will start to move incomplete releases as well.


=== clomps ===


'''* clomps (console multiple site 'whats new?' query program, with auto-queue support)'''
AUTOQ|PASSNUM=2|FROM=ASite1|TO=home | ACCEPT=* | REJECT=*dvdrip*  


[[Image:clomps.screen1.jpg|right|thumb|100px|clomps]]
After pass=1 has completed, move '''everything''' from '''ASite1''' to home. Except for dvdrips. Incompletes will also be started.


This [[FXP.One]] client ships with the FXP.One packages and will be compiled automatically. It is a console only application that connects to multiple servers, works out what entries are '''new''' since it last ran, and presents a matrix of where files/directories are, and are missing. It can also automatically '''mirror''' between sites, in multiple ways, using the pass- and skip- lists. It can setup up sequential mirroring (A->B, then B->C) or concurrent mirroring. (A->B and A->C).


The latest Windows binary is available here (includes clomps binary):
=== IrcBot ===
[http://www.lundman.net/ftp/fxp-one-1.9-win32.zip fxp-one-1.9-win32.zip]


Older:
To define an ircbot to listen to an irc server / irc channel, add the following line:
[http://www.lundman.net/ftp/clomps-v1.0.zip clomps-v1.0.zip]


IRC|server=irc.server.com|port=6667|pass=ircpass|nick=clomps|user=clomps@fxp.one|ssl=yes|channel=#bots


[[clomps_conf|Example]] configuration file for clomps
Optionals are port, pass, ssl. Should we add fishkey?


=== Ixodes ===
When defining queues, we need to be able to specify the matching patterns:  *new release*tv*the.simpsons, or possibly just trigger it to connect, why specify patterns twice?


This [[FXP.One]] client is a GTK GUI client made on Python by a colleague. This client was never released and was made as a test-case for the '''UFxp''' client.
Since we only want one bot for (possibly) many sections, how do we define this? The AUTOQ line will need to have the src and dst dirs specified.


Status: deprecated.
Or should the irc conf be completely different from regular clomps.conf
 
 
=== UFxp ===
 
This [[FXP.One]] client is a Ultimate++ GUI client. This is an external client and the most complete GUI client at the moment. This is most likely what new users wants to use, unless they are in a console only environment. Please see the [[UFxp]] home page for more information.
 
Status: beta, fully functional.
 
[[Image:ufxp.screen1.jpg|right|thumb|100px|UFxp]]

Revision as of 02:52, 23 February 2012

# Which FXP.One to connect to. Omit PASS to have app prompt for passwd.
FXPONE|HOST=server.that.runs.fxpone.com|PORT=8885|USER=admin|PASS=admin|ssl=forced

Define where to connect for the FXP.One engine. It can be the same server as clomps run on, use localhost. If you leave the PASS= out, then clomps will prompt for passwords. ssl=forced on this line refers to the connection to the engine. Encryption is a good idea, except if you are using localhost, and nobody else has root on your server. 

# Define sites and paths.
SITE|NAME=ASite1|DIR=/x264-tv/|USESKIP=1|INCTEST=%s-(INCOMPLETE)|NUKETEST=NUKED-%s|LAST_CHECK=1329887220
SITE|NAME=Another|DIR=/tv-x264/|USESKIP=1|INCTEST=%s-(INCOMPLETE)|NUKETEST=NUKED-%s|LAST_CHECK=1329897720
SITE|NAME=glftpdSite|DIR=/tv-x264/|USESKIP=1|INCTEST=(INCOMPLETE)-%s|NUKETEST=!NUKED-%s|LAST_CHECK=1329919260
SITE|NAME=home|DIR=/tv/|DIR=/drive2/tv/|USESKIP=1|INCTEST=%s-(INCOMPLETE)|NUKETEST=NUKED-%s|LAST_CHECK=1329828420

Here we define the sites to connect to, and which directories to list. Note that home has two directories listed, they are combined. You do not need to specify LAST_CHECK, it is added by clomps automatically. Leave it out if you are creating a new conf. It has the "newest most item's date" stored, so we know what is newer, for each site.

If you run this clomps.conf now, you will get an output like: 

./clomps -vf example.conf

connected to FXP.One
successfully negotiated SSL
Successfully connected to FXP.One
Site glftpdSite processing dir '/tv-x264/'
Site home processing dir '/tv/'
Site Another processing dir '/tv-x264/'
Site ASite1 processing dir '/x264-tv/'
Site home processing dir '/drive2/tv/'
      Another finished processing:  111 files (  1/  4)
         home finished processing: 3135 files (  2/  4)
   glftpdSite finished processing:  158 files (  3/  4)
       ASite1 finished processing:  166 files (  4/  4)


-----======> DISPLAY MATRIX <======-----

T |ASite|Anoth|glftp|home | New Items
--+-----+-----+-----+-----+-----------
d |INC  |MISS |MISS |MISS | Best.Laid.Plans.2012.1080p.BluRay.X264-7SinS
d |INC  |MISS |MISS |MISS | To.Catch.A.Thief.1955.1080p.BluRay.x264-HD4U
d |     |MISS |MISS |MISS | The.Summer.Of.Massacre.2011.720p.BluRay.x264-UNTOUCH
ABLES
d |     |MISS |NUKE |MISS | Under.Still.Waters.2008.720p.BluRay.x264-SWAGGERHD

Which shows new releases, by showing up (names on the right side) they exist somewhere. Any site column with a blank entry, means the release is there and complete. INC means that the release is incomplete (INCTEST patten matches), and similarly NUKE means nuked. MISS means this release is not on this site.


Automatic queueing

In addition to finding new entries, you can ask clomps to create queues. The queue lines are defines with pass number. Starting from 1. This means, all queue lines with pass=1 will be started at the same time. Once all pass=1 transfers are complete, clomps will run pass=2. This allows for all-concurrent queueing, or, sequentially queueing.

For example (Spaces added between '|' separator for readability): 

AUTOQ|PASSNUM=1|FROM=glftpdSite|TO=ASite1 | ACCEPT=the.simpson*/family.guy*/american.dad*/How?I?met?your?mother* | REJECT=*FINSUB*/*AC3* | INCSKIP

Which would automatically transfer items from glftpdSite "/tv-x264/", to ASite1 "/x264-tv/". But only releases matching "the.simpson*" etc, unless they are *FINSUB* etc. The INCSKIP tells clomps not to transfer things that are incomplete. Clomps will then only transfer completed-releases. If you remove INCSKIP clomps will start to move incomplete releases as well. 


AUTOQ|PASSNUM=2|FROM=ASite1|TO=home | ACCEPT=* | REJECT=*dvdrip*

After pass=1 has completed, move everything from ASite1 to home. Except for dvdrips. Incompletes will also be started.


IrcBot

To define an ircbot to listen to an irc server / irc channel, add the following line: 

IRC|server=irc.server.com|port=6667|pass=ircpass|nick=clomps|user=clomps@fxp.one|ssl=yes|channel=#bots

Optionals are port, pass, ssl. Should we add fishkey?

When defining queues, we need to be able to specify the matching patterns: *new release*tv*the.simpsons, or possibly just trigger it to connect, why specify patterns twice?

Since we only want one bot for (possibly) many sections, how do we define this? The AUTOQ line will need to have the src and dst dirs specified.

Or should the irc conf be completely different from regular clomps.conf

Retrieved from "http://lundman.net/wiki/index.php?title=Clomps_conf&oldid=4316"