Batch removing files on a FTP server

Laurent Perrinet

2014-02-28 09:33

In a previous post, I have shown how to convert MoinMoin pages to this blog (Nikola angine). Let's now tidy up the place and remove obsolete pages.

Pages that were succesfully converted:

In [5]:

done_page_list = """\
SciBlog/2008-08-29
"""
page_list = """\
SciBlog/2006-11-20
SciBlog/2006-11-30
SciBlog/2007-10-22
SciBlog/2008-11-20
SciBlog/2008-12-05
SciBlog/2009-08-19
SciBlog/2009-08-20
SciBlog/2009-08-21
SciBlog/2009-08-23
SciBlog/2009-08-26
SciBlog/2009-08-29
SciBlog/2009-11-01
SciBlog/2009-11-15
SciBlog/2009-11-22
SciBlog/2009-11-26
SciBlog/2009-12-02
SciBlog/2009-12-06
SciBlog/2009-12-09
SciBlog/2009-12-11
SciBlog/2009-12-17
SciBlog/2009-12-19
SciBlog/2010-01-27
SciBlog/2010-02-03
SciBlog/2010-02-11
SciBlog/2010-03-15
SciBlog/2010-04-07
SciBlog/2010-04-08
SciBlog/2010-04-24
SciBlog/2010-04-25
SciBlog/2010-04-26
SciBlog/2010-04-28
SciBlog/2010-05-22
SciBlog/2010-05-23
SciBlog/2010-07-08
SciBlog/2010-08-03
SciBlog/2010-08-05
SciBlog/2010-08-07
SciBlog/2010-08-09
SciBlog/2010-08-11
SciBlog/2010-08-29
SciBlog/2010-08-30
SciBlog/2010-08-31
SciBlog/2010-09-03
SciBlog/2010-09-09
SciBlog/2010-09-27
SciBlog/2010-10-05
SciBlog/2010-10-06
SciBlog/2010-10-09
SciBlog/2010-10-27
SciBlog/2010-10-31
SciBlog/2010-11-03
SciBlog/2010-11-04
SciBlog/2010-11-07
SciBlog/2010-11-09
SciBlog/2010-11-15
SciBlog/2010-11-27
SciBlog/2010-12-08
SciBlog/2010-12-20
SciBlog/2010-12-27
SciBlog/2011-01-18
SciBlog/2011-01-19
SciBlog/2011-01-25
SciBlog/2011-01-26
SciBlog/2011-01-29
SciBlog/2011-02-09
SciBlog/2011-02-21
SciBlog/2011-02-22
SciBlog/2011-03-09
SciBlog/2011-03-10
SciBlog/2011-05-04
SciBlog/2011-07-05
SciBlog/2011-07-06
SciBlog/2011-07-07
SciBlog/2011-07-08
SciBlog/2011-07-10
SciBlog/2011-07-25
SciBlog/2012-01-07
SciBlog/2012-01-18
SciBlog/2012-02-08
SciBlog/2012-02-16
SciBlog/2012-02-17
SciBlog/2012-02-22
SciBlog/2012-03-05
SciBlog/2012-03-10
SciBlog/2012-03-11
SciBlog/2012-03-12
SciBlog/2012-03-13
SciBlog/2012-03-16
SciBlog/2012-03-21
SciBlog/2012-03-22
SciBlog/2012-03-24
SciBlog/2012-03-25
SciBlog/2012-04-05
SciBlog/2012-04-17
SciBlog/2012-04-22
SciBlog/2012-04-30
SciBlog/2012-06-12
SciBlog/2012-07-10
SciBlog/2012-07-24
SciBlog/2012-08-19
SciBlog/2012-08-27
SciBlog/2012-09-04
SciBlog/2012-09-09
SciBlog/2012-09-30
SciBlog/2012-11-07
SciBlog/2012-11-28
SciBlog/2013-01-16
SciBlog/2013-01-31
SciBlog/2013-02-02
SciBlog/2013-02-04
SciBlog/2013-03-06
SciBlog/2013-03-13
SciBlog/2013-03-23
SciBlog/2013-03-25
SciBlog/2013-05-14
SciBlog/2013-06-12
SciBlog/2013-07-08
SciBlog/2013-10-26
SciBlog/2013-11-04
SciBlog/2013-11-08
SciBlog/2013-11-09
SciBlog/2014-01-01"""

a bash script to remove one page:

In [17]:

%%bash
lftp -c "
open invibe;
ls -l moin/data/pages/SciBlog(2f)2013(2d)11(2d)04
"

drwxrwx---    4 81424      users               6 Nov 12 10:20 .
drwxr-xr-x  884 81424      users             884 Mar 18 09:16 ..
drwxrwx---    2 81424      users               4 Nov 12 10:20 cache
-rw-rw----    1 81424      users               9 Nov 12 10:20 current
-rw-rw----    1 81424      users             366 Nov 12 10:20 edit-log
drwxrwx---    2 81424      users               5 Nov 12 10:20 revisions

creating a simple function that will run a command given by a string:

In [15]:

%%bash
lftp -c "open invibe; ls moin/data/pages/SciBlog(2f)2010(2d)12(2d)20 ;"

drwxr-xr-x    4 81424      users               6 Mar 13  2012 .
drwxr-xr-x  884 81424      users             884 Mar 18 09:16 ..
drwxr-xr-x    2 81424      users               4 Jun 19  2013 cache
-rwxr-xr-x    1 81424      users               9 Mar 13  2012 current
-rwxr-xr-x    1 81424      users             895 Mar 13  2012 edit-log
drwxr-xr-x    2 81424      users               9 Mar 13  2012 revisions

In [32]:

cmd = 'lftp -c "open invibe; cd moin/data/pages/ ; \n'
for page in page_list.split('\n'):
    cmd +=  "rm -fr " + page.replace('-', '(2d)').replace('/', '(2f)')
    cmd += ' \n'
cmd += '\n"'
print cmd

lftp -c "open invibe; cd moin/data/pages/ ; 
rm -fr SciBlog(2f)2006(2d)11(2d)20 
rm -fr SciBlog(2f)2006(2d)11(2d)30 
rm -fr SciBlog(2f)2007(2d)10(2d)22 
rm -fr SciBlog(2f)2008(2d)11(2d)20 
rm -fr SciBlog(2f)2008(2d)12(2d)05 
rm -fr SciBlog(2f)2009(2d)08(2d)19 
rm -fr SciBlog(2f)2009(2d)08(2d)20 
rm -fr SciBlog(2f)2009(2d)08(2d)21 
rm -fr SciBlog(2f)2009(2d)08(2d)23 
rm -fr SciBlog(2f)2009(2d)08(2d)26 
rm -fr SciBlog(2f)2009(2d)08(2d)29 
rm -fr SciBlog(2f)2009(2d)11(2d)01 
rm -fr SciBlog(2f)2009(2d)11(2d)15 
rm -fr SciBlog(2f)2009(2d)11(2d)22 
rm -fr SciBlog(2f)2009(2d)11(2d)26 
rm -fr SciBlog(2f)2009(2d)12(2d)02 
rm -fr SciBlog(2f)2009(2d)12(2d)06 
rm -fr SciBlog(2f)2009(2d)12(2d)09 
rm -fr SciBlog(2f)2009(2d)12(2d)11 
rm -fr SciBlog(2f)2009(2d)12(2d)17 
rm -fr SciBlog(2f)2009(2d)12(2d)19 
rm -fr SciBlog(2f)2010(2d)01(2d)27 
rm -fr SciBlog(2f)2010(2d)02(2d)03 
rm -fr SciBlog(2f)2010(2d)02(2d)11 
rm -fr SciBlog(2f)2010(2d)03(2d)15 
rm -fr SciBlog(2f)2010(2d)04(2d)07 
rm -fr SciBlog(2f)2010(2d)04(2d)08 
rm -fr SciBlog(2f)2010(2d)04(2d)24 
rm -fr SciBlog(2f)2010(2d)04(2d)25 
rm -fr SciBlog(2f)2010(2d)04(2d)26 
rm -fr SciBlog(2f)2010(2d)04(2d)28 
rm -fr SciBlog(2f)2010(2d)05(2d)22 
rm -fr SciBlog(2f)2010(2d)05(2d)23 
rm -fr SciBlog(2f)2010(2d)07(2d)08 
rm -fr SciBlog(2f)2010(2d)08(2d)03 
rm -fr SciBlog(2f)2010(2d)08(2d)05 
rm -fr SciBlog(2f)2010(2d)08(2d)07 
rm -fr SciBlog(2f)2010(2d)08(2d)09 
rm -fr SciBlog(2f)2010(2d)08(2d)11 
rm -fr SciBlog(2f)2010(2d)08(2d)29 
rm -fr SciBlog(2f)2010(2d)08(2d)30 
rm -fr SciBlog(2f)2010(2d)08(2d)31 
rm -fr SciBlog(2f)2010(2d)09(2d)03 
rm -fr SciBlog(2f)2010(2d)09(2d)09 
rm -fr SciBlog(2f)2010(2d)09(2d)27 
rm -fr SciBlog(2f)2010(2d)10(2d)05 
rm -fr SciBlog(2f)2010(2d)10(2d)06 
rm -fr SciBlog(2f)2010(2d)10(2d)09 
rm -fr SciBlog(2f)2010(2d)10(2d)27 
rm -fr SciBlog(2f)2010(2d)10(2d)31 
rm -fr SciBlog(2f)2010(2d)11(2d)03 
rm -fr SciBlog(2f)2010(2d)11(2d)04 
rm -fr SciBlog(2f)2010(2d)11(2d)07 
rm -fr SciBlog(2f)2010(2d)11(2d)09 
rm -fr SciBlog(2f)2010(2d)11(2d)15 
rm -fr SciBlog(2f)2010(2d)11(2d)27 
rm -fr SciBlog(2f)2010(2d)12(2d)08 
rm -fr SciBlog(2f)2010(2d)12(2d)20 
rm -fr SciBlog(2f)2010(2d)12(2d)27 
rm -fr SciBlog(2f)2011(2d)01(2d)18 
rm -fr SciBlog(2f)2011(2d)01(2d)19 
rm -fr SciBlog(2f)2011(2d)01(2d)25 
rm -fr SciBlog(2f)2011(2d)01(2d)26 
rm -fr SciBlog(2f)2011(2d)01(2d)29 
rm -fr SciBlog(2f)2011(2d)02(2d)09 
rm -fr SciBlog(2f)2011(2d)02(2d)21 
rm -fr SciBlog(2f)2011(2d)02(2d)22 
rm -fr SciBlog(2f)2011(2d)03(2d)09 
rm -fr SciBlog(2f)2011(2d)03(2d)10 
rm -fr SciBlog(2f)2011(2d)05(2d)04 
rm -fr SciBlog(2f)2011(2d)07(2d)05 
rm -fr SciBlog(2f)2011(2d)07(2d)06 
rm -fr SciBlog(2f)2011(2d)07(2d)07 
rm -fr SciBlog(2f)2011(2d)07(2d)08 
rm -fr SciBlog(2f)2011(2d)07(2d)10 
rm -fr SciBlog(2f)2011(2d)07(2d)25 
rm -fr SciBlog(2f)2012(2d)01(2d)07 
rm -fr SciBlog(2f)2012(2d)01(2d)18 
rm -fr SciBlog(2f)2012(2d)02(2d)08 
rm -fr SciBlog(2f)2012(2d)02(2d)16 
rm -fr SciBlog(2f)2012(2d)02(2d)17 
rm -fr SciBlog(2f)2012(2d)02(2d)22 
rm -fr SciBlog(2f)2012(2d)03(2d)05 
rm -fr SciBlog(2f)2012(2d)03(2d)10 
rm -fr SciBlog(2f)2012(2d)03(2d)11 
rm -fr SciBlog(2f)2012(2d)03(2d)12 
rm -fr SciBlog(2f)2012(2d)03(2d)13 
rm -fr SciBlog(2f)2012(2d)03(2d)16 
rm -fr SciBlog(2f)2012(2d)03(2d)21 
rm -fr SciBlog(2f)2012(2d)03(2d)22 
rm -fr SciBlog(2f)2012(2d)03(2d)24 
rm -fr SciBlog(2f)2012(2d)03(2d)25 
rm -fr SciBlog(2f)2012(2d)04(2d)05 
rm -fr SciBlog(2f)2012(2d)04(2d)17 
rm -fr SciBlog(2f)2012(2d)04(2d)22 
rm -fr SciBlog(2f)2012(2d)04(2d)30 
rm -fr SciBlog(2f)2012(2d)06(2d)12 
rm -fr SciBlog(2f)2012(2d)07(2d)10 
rm -fr SciBlog(2f)2012(2d)07(2d)24 
rm -fr SciBlog(2f)2012(2d)08(2d)19 
rm -fr SciBlog(2f)2012(2d)08(2d)27 
rm -fr SciBlog(2f)2012(2d)09(2d)04 
rm -fr SciBlog(2f)2012(2d)09(2d)09 
rm -fr SciBlog(2f)2012(2d)09(2d)30 
rm -fr SciBlog(2f)2012(2d)11(2d)07 
rm -fr SciBlog(2f)2012(2d)11(2d)28 
rm -fr SciBlog(2f)2013(2d)01(2d)16 
rm -fr SciBlog(2f)2013(2d)01(2d)31 
rm -fr SciBlog(2f)2013(2d)02(2d)02 
rm -fr SciBlog(2f)2013(2d)02(2d)04 
rm -fr SciBlog(2f)2013(2d)03(2d)06 
rm -fr SciBlog(2f)2013(2d)03(2d)13 
rm -fr SciBlog(2f)2013(2d)03(2d)23 
rm -fr SciBlog(2f)2013(2d)03(2d)25 
rm -fr SciBlog(2f)2013(2d)05(2d)14 
rm -fr SciBlog(2f)2013(2d)06(2d)12 
rm -fr SciBlog(2f)2013(2d)07(2d)08 
rm -fr SciBlog(2f)2013(2d)10(2d)26 
rm -fr SciBlog(2f)2013(2d)11(2d)04 
rm -fr SciBlog(2f)2013(2d)11(2d)08 
rm -fr SciBlog(2f)2013(2d)11(2d)09 
rm -fr SciBlog(2f)2014(2d)01(2d)01 

"

Actually doing it:

In [33]:

%%bash
lftp -c "open invibe; cd moin/data/pages/ ; 
rm -fr SciBlog(2f)2006(2d)11(2d)20 
rm -fr SciBlog(2f)2006(2d)11(2d)30 
rm -fr SciBlog(2f)2007(2d)10(2d)22 
rm -fr SciBlog(2f)2008(2d)11(2d)20 
rm -fr SciBlog(2f)2008(2d)12(2d)05 
rm -fr SciBlog(2f)2009(2d)08(2d)19 
rm -fr SciBlog(2f)2009(2d)08(2d)20 
rm -fr SciBlog(2f)2009(2d)08(2d)21 
rm -fr SciBlog(2f)2009(2d)08(2d)23 
rm -fr SciBlog(2f)2009(2d)08(2d)26 
rm -fr SciBlog(2f)2009(2d)08(2d)29 
rm -fr SciBlog(2f)2009(2d)11(2d)01 
rm -fr SciBlog(2f)2009(2d)11(2d)15 
rm -fr SciBlog(2f)2009(2d)11(2d)22 
rm -fr SciBlog(2f)2009(2d)11(2d)26 
rm -fr SciBlog(2f)2009(2d)12(2d)02 
rm -fr SciBlog(2f)2009(2d)12(2d)06 
rm -fr SciBlog(2f)2009(2d)12(2d)09 
rm -fr SciBlog(2f)2009(2d)12(2d)11 
rm -fr SciBlog(2f)2009(2d)12(2d)17 
rm -fr SciBlog(2f)2009(2d)12(2d)19 
rm -fr SciBlog(2f)2010(2d)01(2d)27 
rm -fr SciBlog(2f)2010(2d)02(2d)03 
rm -fr SciBlog(2f)2010(2d)02(2d)11 
rm -fr SciBlog(2f)2010(2d)03(2d)15 
rm -fr SciBlog(2f)2010(2d)04(2d)07 
rm -fr SciBlog(2f)2010(2d)04(2d)08 
rm -fr SciBlog(2f)2010(2d)04(2d)24 
rm -fr SciBlog(2f)2010(2d)04(2d)25 
rm -fr SciBlog(2f)2010(2d)04(2d)26 
rm -fr SciBlog(2f)2010(2d)04(2d)28 
rm -fr SciBlog(2f)2010(2d)05(2d)22 
rm -fr SciBlog(2f)2010(2d)05(2d)23 
rm -fr SciBlog(2f)2010(2d)07(2d)08 
rm -fr SciBlog(2f)2010(2d)08(2d)03 
rm -fr SciBlog(2f)2010(2d)08(2d)05 
rm -fr SciBlog(2f)2010(2d)08(2d)07 
rm -fr SciBlog(2f)2010(2d)08(2d)09 
rm -fr SciBlog(2f)2010(2d)08(2d)11 
rm -fr SciBlog(2f)2010(2d)08(2d)29 
rm -fr SciBlog(2f)2010(2d)08(2d)30 
rm -fr SciBlog(2f)2010(2d)08(2d)31 
rm -fr SciBlog(2f)2010(2d)09(2d)03 
rm -fr SciBlog(2f)2010(2d)09(2d)09 
rm -fr SciBlog(2f)2010(2d)09(2d)27 
rm -fr SciBlog(2f)2010(2d)10(2d)05 
rm -fr SciBlog(2f)2010(2d)10(2d)06 
rm -fr SciBlog(2f)2010(2d)10(2d)09 
rm -fr SciBlog(2f)2010(2d)10(2d)27 
rm -fr SciBlog(2f)2010(2d)10(2d)31 
rm -fr SciBlog(2f)2010(2d)11(2d)03 
rm -fr SciBlog(2f)2010(2d)11(2d)04 
rm -fr SciBlog(2f)2010(2d)11(2d)07 
rm -fr SciBlog(2f)2010(2d)11(2d)09 
rm -fr SciBlog(2f)2010(2d)11(2d)15 
rm -fr SciBlog(2f)2010(2d)11(2d)27 
rm -fr SciBlog(2f)2010(2d)12(2d)08 
rm -fr SciBlog(2f)2010(2d)12(2d)20 
rm -fr SciBlog(2f)2010(2d)12(2d)27 
rm -fr SciBlog(2f)2011(2d)01(2d)18 
rm -fr SciBlog(2f)2011(2d)01(2d)19 
rm -fr SciBlog(2f)2011(2d)01(2d)25 
rm -fr SciBlog(2f)2011(2d)01(2d)26 
rm -fr SciBlog(2f)2011(2d)01(2d)29 
rm -fr SciBlog(2f)2011(2d)02(2d)09 
rm -fr SciBlog(2f)2011(2d)02(2d)21 
rm -fr SciBlog(2f)2011(2d)02(2d)22 
rm -fr SciBlog(2f)2011(2d)03(2d)09 
rm -fr SciBlog(2f)2011(2d)03(2d)10 
rm -fr SciBlog(2f)2011(2d)05(2d)04 
rm -fr SciBlog(2f)2011(2d)07(2d)05 
rm -fr SciBlog(2f)2011(2d)07(2d)06 
rm -fr SciBlog(2f)2011(2d)07(2d)07 
rm -fr SciBlog(2f)2011(2d)07(2d)08 
rm -fr SciBlog(2f)2011(2d)07(2d)10 
rm -fr SciBlog(2f)2011(2d)07(2d)25 
rm -fr SciBlog(2f)2012(2d)01(2d)07 
rm -fr SciBlog(2f)2012(2d)01(2d)18 
rm -fr SciBlog(2f)2012(2d)02(2d)08 
rm -fr SciBlog(2f)2012(2d)02(2d)16 
rm -fr SciBlog(2f)2012(2d)02(2d)17 
rm -fr SciBlog(2f)2012(2d)02(2d)22 
rm -fr SciBlog(2f)2012(2d)03(2d)05 
rm -fr SciBlog(2f)2012(2d)03(2d)10 
rm -fr SciBlog(2f)2012(2d)03(2d)11 
rm -fr SciBlog(2f)2012(2d)03(2d)12 
rm -fr SciBlog(2f)2012(2d)03(2d)13 
rm -fr SciBlog(2f)2012(2d)03(2d)16 
rm -fr SciBlog(2f)2012(2d)03(2d)21 
rm -fr SciBlog(2f)2012(2d)03(2d)22 
rm -fr SciBlog(2f)2012(2d)03(2d)24 
rm -fr SciBlog(2f)2012(2d)03(2d)25 
rm -fr SciBlog(2f)2012(2d)04(2d)05 
rm -fr SciBlog(2f)2012(2d)04(2d)17 
rm -fr SciBlog(2f)2012(2d)04(2d)22 
rm -fr SciBlog(2f)2012(2d)04(2d)30 
rm -fr SciBlog(2f)2012(2d)06(2d)12 
rm -fr SciBlog(2f)2012(2d)07(2d)10 
rm -fr SciBlog(2f)2012(2d)07(2d)24 
rm -fr SciBlog(2f)2012(2d)08(2d)19 
rm -fr SciBlog(2f)2012(2d)08(2d)27 
rm -fr SciBlog(2f)2012(2d)09(2d)04 
rm -fr SciBlog(2f)2012(2d)09(2d)09 
rm -fr SciBlog(2f)2012(2d)09(2d)30 
rm -fr SciBlog(2f)2012(2d)11(2d)07 
rm -fr SciBlog(2f)2012(2d)11(2d)28 
rm -fr SciBlog(2f)2013(2d)01(2d)16 
rm -fr SciBlog(2f)2013(2d)01(2d)31 
rm -fr SciBlog(2f)2013(2d)02(2d)02 
rm -fr SciBlog(2f)2013(2d)02(2d)04 
rm -fr SciBlog(2f)2013(2d)03(2d)06 
rm -fr SciBlog(2f)2013(2d)03(2d)13 
rm -fr SciBlog(2f)2013(2d)03(2d)23 
rm -fr SciBlog(2f)2013(2d)03(2d)25 
rm -fr SciBlog(2f)2013(2d)05(2d)14 
rm -fr SciBlog(2f)2013(2d)06(2d)12 
rm -fr SciBlog(2f)2013(2d)07(2d)08 
rm -fr SciBlog(2f)2013(2d)10(2d)26 
rm -fr SciBlog(2f)2013(2d)11(2d)04 
rm -fr SciBlog(2f)2013(2d)11(2d)08 
rm -fr SciBlog(2f)2013(2d)11(2d)09 
rm -fr SciBlog(2f)2014(2d)01(2d)01 

"