Converting moinmoin pages to nikola

Laurent Perrinet

2014-02-15 15:09

nikolFrom a previous post, we have this function to import a page from moinmoin, convert it and publish:

In [1]:

def url2rst(URL):
    import urllib
    import subprocess
    f = urllib.urlopen(URL + '?action=print')
    html = f.read()
    p = subprocess.Popen(['pandoc', '--from=html', '--to=rst'],
                         stdin=subprocess.PIPE, stdout=subprocess.PIPE)
    return p.communicate(html)[0]

In [2]:

def convert(URL, DATE, TIME, TITLE, TAGS, doit=True):
    pagerst = url2rst(URL)
  
    TIMEDTITLE = DATE[2:] + ' ' + TITLE
    SLUG = DATE[2:] + '-' + TITLE.replace(' ', '-')
    tmplt = """\
.. title: %s %s
.. slug: %s
.. date: %s %s
.. type: text
.. tags: %s
    
""" % (DATE[2:], TITLE, SLUG, DATE, TIME, TAGS)
    
    if doit:
        import os
        print TITLE, TAGS
        cmd = 'cd ..; nikola new_post --title="%s" --tags="%s"' % (TIMEDTITLE, TAGS)
        print cmd
        os.system(cmd)
        f = file(SLUG + '.rst', 'w')
        f.write(tmplt + pagerst)
        f.close()
    else:
        print tmplt

In [3]:

URL = 'https://invibe.net/LaurentPerrinet/Presentations/10-10-20_M2_MasterSciences'
DATE, TIME = '2010-10-20', '13:36:57'
TITLE, TAGS = 'Master M2 Sciences', 'Talks, ComputationalNeuroscience, sciblog'
#convert(URL, DATE, TIME, TITLE, TAGS, doit=False)

In [4]:

URL = 'https://invibe.net/LaurentPerrinet/Presentations/10-10-27_M2_MasterSciences'
DATE, TIME = '2010-10-27', '13:36:57'
TITLE, TAGS = 'Master M2 Sciences', 'Talks, ComputationalNeuroscience, sciblog'
#convert(URL, DATE, TIME, TITLE, TAGS, doit=False)

extracting title and tags¶

In [5]:

def url2raw(URL):
    import urllib
    import subprocess
    f = urllib.urlopen(URL + '?action=raw')
    raw = f.read()
    return raw

In [6]:

DATE, TIME = '2009-08-20', '13:36:57'
URL = 'https://invibe.net/LaurentPerrinet/SciBlog/' + DATE
RAW = url2raw(URL)
#print RAW

Some heuristics to extract the title (always a level 1 header):

In [7]:

#import string
for line in RAW.split('\n'):
    if len(line)>2: 
        if line[0] == '=':
            if line[0:2] == '= ': 
                TITLE = line[2:-2]
                TITLE = TITLE.strip('\r')
                print "%r, %r" % (line, TITLE)
                #for char in TITLE: print char
                print ',' in TITLE
                # while TITLE[-1]==' ': TITLE = TITLE[:-1]
                #TITLE.replace?
                #TITLE = unicode(TITLE, "utf-8")
                TITLE = TITLE.replace(',', '-')
                print "%r, %r" % (line, TITLE)
                TITLE = TITLE.replace('drafts', 'toto')
                print "%r, %r" % (line, TITLE)
import re
TITLE = re.sub(',', '-', TITLE)
#TITLE = str(TITLE)
#TITLE.replace(u",", u"-")
print type(TITLE), TITLE

'= some LaTeX tips: drafts, links, margins, pdflatex =\r', 'some LaTeX tips: drafts, links, margins, pdflatex '
True
'= some LaTeX tips: drafts, links, margins, pdflatex =\r', 'some LaTeX tips: drafts- links- margins- pdflatex '
'= some LaTeX tips: drafts, links, margins, pdflatex =\r', 'some LaTeX tips: toto- links- margins- pdflatex '
<type 'str'> some LaTeX tips: toto- links- margins- pdflatex

In [8]:

print 'some LaTeX tips: drafts / links / margins / pdflatex'.replace('/', '-')
TITLE = 'some LaTeX tips: drafts / links / margins / pdflatex'
exclude_chars =[':', ' :', ': ', '/', '  ', '--']
for char in exclude_chars:
    TITLE = TITLE.replace(char, '-')
print TITLE

some LaTeX tips: drafts - links - margins - pdflatex
some LaTeX tips- drafts - links - margins - pdflatex

Some heuristics to extract the tags (always begins with "Tags"):

In [9]:

TAGS = []
for line in RAW.split('\n'):
    for word in line.split(' '):
        if len(word)>3: 
            if word[0:3] == 'Tag': 
                TAGS.append(word[3:])
print ', '.join(TAGS)

LaTex

Some heuristics to process the rst code:

In [10]:

pagerst = url2rst(URL)

In [11]:

#import re
pagerst_ = ''
line_old = ''
for i_line, line in enumerate(pagerst.split('\n')):
    if i_line >2 :
        if not line[:4]=='`Tag':
            if not (line_old=='' and line[:6]=='------'):
                line = line.replace('/moin_static196/', 'https://invibe.net/moin_static196/')
                pagerst_ += line + '\n'
    else:
        print 'stripping title :', line
    line_old = line
print pagerst_
#https://invibe.net/moin_static196/moniker/img/angry.png

stripping title : some LaTeX tips: drafts, links, margins, pdflatex
stripping title : =================================================
stripping title : 
checking typographic style
--------------------------

-  Lint for LaTeX :
   `http://baruch.ev-en.org/proj/chktex/ <http://baruch.ev-en.org/proj/chktex/>`__

make links
----------

-  hyperref quick reference list :

   ::

       \href{URL}{text } 
       \url{URL}
       \nolinkurl{URL}

managing margins
----------------

-  to adjust margins, use

   ::

       \usepackage[margin=2.5cm]{geometry}

   then play around with the ``2.5cm`` value until it fits.

-  tips for fitting your text in the required size : `LaTeX Tips n
   Tricks for Conference
   Paper <http://www-db.stanford.edu/~manku/latex.html>`__

citations
---------

-  If you give LaTeX \\cite{fred,joe,harry,min}, its default commands
   could give something like "[2,6,4,3]"; this looks awful. One can of
   course get the things in order by rearranging the keys in the \\cite
   command, but who wants to do that sort of thing for no more
   improvement than "[2,3,4,6]"

   -  The cite package sorts the numbers and detects consecutive
      sequences, so creating "[2-4,6]". The natbib package, with the
      numbers and sort&compress options, will do the same when working
      with its own numeric bibliography styles (plainnat.bst and
      unsrtnat.bst).
   -  If you might need to make hyperreferences to your citations, cite
      isn't adequate. If you add the hypernat package:

      ::

            \usepackage[...]{hyperref}
            \usepackage[numbers,sort&compress]{natbib}
            \usepackage{hypernat}
            ...
            \bibliographystyle{plainnat}

      See for example
      `http://www.tex.ac.uk/cgi-bin/texfaq2html?label=citesort <http://www.tex.ac.uk/cgi-bin/texfaq2html?label=citesort>`__

Useful draft tips
-----------------

-  “LaTeX and Subversion”

   -  set a keyword with ``svn propset svn:keywords "Id" index.tex `` so
      that every occurrence of
   -  use latex-svninfo
      `http://www.ctan.org/tex-archive/macros/latex/contrib/svninfo/ <http://www.ctan.org/tex-archive/macros/latex/contrib/svninfo/>`__
      for instance with

      ::

          \usepackage[fancyhdr,today,draft]{svninfo}%
          %\usepackage[fancyhdr]{svninfo}%
          \pagestyle{fancyplain}
          \fancyhead{}

   -  now at every commit $Id$ will be replaced by useful data that will
      show up in the foot of the page
   -  for a reference on using keywords see
      `http://wiki.loria.fr/wiki/Variables\_automatiques <http://wiki.loria.fr/wiki/Variables_automatiques>`__

using pdfLaTeX
--------------

-  |X-(| sites like arXiV use only plain LaTeX so that you should keep
   the 2 versions of your directives for better portability (see \\ifpdf
   ...)

   -  in particular arXiV rejects the ``microtype`` package

-  le package hyperref permet même de faire des références vers les
   différents chapitres.
-  PDfLaTeX ne permet pas d'inclure des eps pour cela il faut les
   convertir en pdf avec
   `epstopdf <http://www.ctan.org/tex-archive/support/epstopdf/>`__ ou
   le script suivant qui permet de convertir tous les .eps d'un dossier
   (à sauver et rendre executable):

   ::

       for f in $* ;do
           if echo "$f" | grep -i eps*   ; then
                epstopdf --nocompress $f
                echo "converting  $f to pdf ..."
           else
           echo "$f is not a eps file, ignored"
           fi
       done

   il suffit alors d'executer en console
   `` ./mon_script la-ou-ya-tout-mes-eps/*.eps ``



.. |X-(| image:: https://invibe.net/moin_static196/moniker/img/angry.png

wrapping-up:

In [12]:

def extract(URL):
    import re
    RAW = url2raw(URL)
    TITLE = "Unknown title"
    TAGS = []
    for line in RAW.split("\n"):
        if len(line)>2: 
            if line[0:2] == "= ": 
                TITLE = line[2:-2].strip()
                TITLE.strip()
                exclude_chars =[' / ', ' /', '/ ', '/', '  ']
                for char in exclude_chars:
                    TITLE = re.sub(char, '-', TITLE)
        for word in line.split(" "):
            if len(word)>3: 
                if word[0:3] == "Tag": 
                    TAGS.append(word[3:].strip('\r').lower())
    #if not('sciblog' in TAGS): TAGS.append('sciblog')
    TAGS = set(TAGS)
    return TITLE, ", ".join(TAGS)

the following function includes all of the above plus a hack to remove a former version of the script that created filenames with special characters.

In [58]:

def convertSciBlog(DATE, TIME='13:36:57', doit=True):
    URL = 'https://invibe.net/LaurentPerrinet/SciBlog/' + DATE
    TITLE, TAGS = extract(URL)
    #print URL, TITLE, TAGS
    pagerst = url2rst(URL)

    pagerst_parsed = ''
    line_old = ''
    finished_title = False
    for line in pagerst.split('\n'):
        if finished_title :
            if not line[:4]=='`Tag':
                if not (line_old=='' and line[:6]=='------'):
                    line = line.replace('/moin_static196/', 'https://invibe.net/moin_static196/')
                    line = line.replace('</LaurentPerrinet/', '<https://invibe.net/LaurentPerrinet/')
                    if 'SciBlog' in line: print 'Warning ', line
                    pagerst_parsed += line + '\n'
        else:
            if line[:4]=='====': finished_title = True
            print 'stripping title :', line
        line_old = line
  
    TIMEDTITLE = DATE[2:] + ' ' + TITLE

    # The following is a hack to remove old files which had improper characters
    #SLUG_OLD = DATE[2:] + '-' + TITLE.replace(' ', '-')
    SLUG = DATE[2:] + '-' + TITLE
    exclude_chars, exclude_chars_space =[':', '"', ',', ' ', '!', '--', '--'], []
    for char in exclude_chars:
        exclude_chars_space.append(char)
        exclude_chars_space.append(' ' + char)
        exclude_chars_space.append(char + ' ')
        exclude_chars_space.append(' ' + char + ' ')
    for char in exclude_chars_space:
        SLUG = re.sub(char, '-', SLUG)

    translate = {'é':'e', 'è':'e', 'ê':'e', 'à':'a', 'á': 'a', 'É': 'E', "'": '', "\.": ''}
    for char in translate.keys():
        SLUG = re.sub(char, translate[char], SLUG)
                    
    tmplt = """\
.. title: %s %s
.. slug: %s
.. date: %s %s
.. type: text
.. tags: %s

""" % (DATE[2:], TITLE, SLUG, DATE, TIME, TAGS)
    if doit:
        import os
        #print URL, TITLE, TAGS
        print TIMEDTITLE, TAGS
        f = file(SLUG + '.rst', 'w')
        f.write(tmplt + pagerst_parsed)
        f.close()
    else:
        print tmplt

In [59]:

DATE = '2010-10-06'
URL = 'https://invibe.net/LaurentPerrinet/SciBlog/' + DATE
TITLE, TAGS = extract(URL)
TIMEDTITLE = DATE[2:] + ' ' + TITLE
print TIMEDTITLE, TAGS

10-10-06 Émergence sciblog

In [60]:

print 'nikola new_post --title="%r" --tags="%r" ' % (DATE[2:] + ' ' + TITLE, TAGS)

nikola new_post --title="'10-10-06 \xc3\x89mergence'" --tags="'sciblog'"

In [61]:

convertSciBlog(DATE, doit=False)

stripping title : --------------
stripping title : 
stripping title : Émergence
stripping title : =========
.. title: 10-10-06 Émergence
.. slug: 10-10-06-Emergence
.. date: 2010-10-06 13:36:57
.. type: text
.. tags: sciblog

In [62]:

convertSciBlog(DATE, doit=True)

stripping title : --------------
stripping title : 
stripping title : Émergence
stripping title : =========
10-10-06 Émergence sciblog

In [55]:

done_page_list = """\
"""
page_list = """\
SciBlog/2006-11-20
SciBlog/2006-11-30
SciBlog/2007-10-22
SciBlog/2008-08-29
SciBlog/2008-11-20
SciBlog/2008-12-05
SciBlog/2009-08-19
SciBlog/2009-08-20
SciBlog/2009-08-21
SciBlog/2009-08-23
SciBlog/2009-08-26
SciBlog/2009-08-29
SciBlog/2009-11-01
SciBlog/2009-11-15
SciBlog/2009-11-22
SciBlog/2009-11-26
SciBlog/2009-12-02
SciBlog/2009-12-06
SciBlog/2009-12-09
SciBlog/2009-12-11
SciBlog/2009-12-17
SciBlog/2009-12-19
SciBlog/2010-01-27
SciBlog/2010-02-03
SciBlog/2010-02-11
SciBlog/2010-03-15
SciBlog/2010-04-07
SciBlog/2010-04-08
SciBlog/2010-04-24
SciBlog/2010-04-25
SciBlog/2010-04-26
SciBlog/2010-04-28
SciBlog/2010-05-22
SciBlog/2010-05-23
SciBlog/2010-07-08
SciBlog/2010-08-03
SciBlog/2010-08-05
SciBlog/2010-08-07
SciBlog/2010-08-09
SciBlog/2010-08-11
SciBlog/2010-08-29
SciBlog/2010-08-30
SciBlog/2010-08-31
SciBlog/2010-09-03
SciBlog/2010-09-09
SciBlog/2010-09-27
SciBlog/2010-10-05
SciBlog/2010-10-06
SciBlog/2010-10-09
SciBlog/2010-10-27
SciBlog/2010-10-31
SciBlog/2010-11-03
SciBlog/2010-11-04
SciBlog/2010-11-07
SciBlog/2010-11-09
SciBlog/2010-11-15
SciBlog/2010-11-27
SciBlog/2010-12-08
SciBlog/2010-12-20
SciBlog/2010-12-27
SciBlog/2011-01-18
SciBlog/2011-01-19
SciBlog/2011-01-25
SciBlog/2011-01-26
SciBlog/2011-01-29
SciBlog/2011-02-09
SciBlog/2011-02-21
SciBlog/2011-02-22
SciBlog/2011-03-09
SciBlog/2011-03-10
SciBlog/2011-05-04
SciBlog/2011-07-05
SciBlog/2011-07-06
SciBlog/2011-07-07
SciBlog/2011-07-08
SciBlog/2011-07-10
SciBlog/2011-07-25
SciBlog/2012-01-07
SciBlog/2012-01-18
SciBlog/2012-02-08
SciBlog/2012-02-16
SciBlog/2012-02-17
SciBlog/2012-02-22
SciBlog/2012-03-05
SciBlog/2012-03-10
SciBlog/2012-03-11
SciBlog/2012-03-12
SciBlog/2012-03-13
SciBlog/2012-03-16
SciBlog/2012-03-21
SciBlog/2012-03-22
SciBlog/2012-03-24
SciBlog/2012-03-25
SciBlog/2012-04-05
SciBlog/2012-04-17
SciBlog/2012-04-22
SciBlog/2012-04-30
SciBlog/2012-06-12
SciBlog/2012-07-10
SciBlog/2012-07-24
SciBlog/2012-08-19
SciBlog/2012-08-27
SciBlog/2012-09-04
SciBlog/2012-09-09
SciBlog/2012-09-30
SciBlog/2012-11-07
SciBlog/2012-11-28
SciBlog/2013-01-16
SciBlog/2013-01-31
SciBlog/2013-02-02
SciBlog/2013-02-04
SciBlog/2013-03-06
SciBlog/2013-03-13
SciBlog/2013-03-23
SciBlog/2013-03-25
SciBlog/2013-05-14
SciBlog/2013-06-12
SciBlog/2013-07-08
SciBlog/2013-10-26
SciBlog/2013-11-04
SciBlog/2013-11-08
SciBlog/2013-11-09
SciBlog/2014-01-01"""

In [56]:

for page in page_list.split('\n'):
    print page.replace('SciBlog/', ''),

2006-11-20 2006-11-30 2007-10-22 2008-08-29 2008-11-20 2008-12-05 2009-08-19 2009-08-20 2009-08-21 2009-08-23 2009-08-26 2009-08-29 2009-11-01 2009-11-15 2009-11-22 2009-11-26 2009-12-02 2009-12-06 2009-12-09 2009-12-11 2009-12-17 2009-12-19 2010-01-27 2010-02-03 2010-02-11 2010-03-15 2010-04-07 2010-04-08 2010-04-24 2010-04-25 2010-04-26 2010-04-28 2010-05-22 2010-05-23 2010-07-08 2010-08-03 2010-08-05 2010-08-07 2010-08-09 2010-08-11 2010-08-29 2010-08-30 2010-08-31 2010-09-03 2010-09-09 2010-09-27 2010-10-05 2010-10-06 2010-10-09 2010-10-27 2010-10-31 2010-11-03 2010-11-04 2010-11-07 2010-11-09 2010-11-15 2010-11-27 2010-12-08 2010-12-20 2010-12-27 2011-01-18 2011-01-19 2011-01-25 2011-01-26 2011-01-29 2011-02-09 2011-02-21 2011-02-22 2011-03-09 2011-03-10 2011-05-04 2011-07-05 2011-07-06 2011-07-07 2011-07-08 2011-07-10 2011-07-25 2012-01-07 2012-01-18 2012-02-08 2012-02-16 2012-02-17 2012-02-22 2012-03-05 2012-03-10 2012-03-11 2012-03-12 2012-03-13 2012-03-16 2012-03-21 2012-03-22 2012-03-24 2012-03-25 2012-04-05 2012-04-17 2012-04-22 2012-04-30 2012-06-12 2012-07-10 2012-07-24 2012-08-19 2012-08-27 2012-09-04 2012-09-09 2012-09-30 2012-11-07 2012-11-28 2013-01-16 2013-01-31 2013-02-02 2013-02-04 2013-03-06 2013-03-13 2013-03-23 2013-03-25 2013-05-14 2013-06-12 2013-07-08 2013-10-26 2013-11-04 2013-11-08 2013-11-09 2014-01-01

In [57]:

for page in page_list.split('\n'):
    convertSciBlog(DATE=page.replace('SciBlog/', ''), doit=True)

stripping title : --------------
stripping title : 
stripping title : V1 hypercolumn Coordination Meeting, 20th - 21st Nov 2006
stripping title : =========================================================
06-11-20 V1 hypercolumn Coordination Meeting, 20th - 21st Nov 2006 facets, sciblog
stripping title : Post-doctoral Position in Computational Neuroscience: "Functional, Large-scale Models of Visual Motion Perception"
stripping title : ==================================================================================================================
06-11-30 Post-doctoral Position in Computational Neuroscience: "Functional, Large-scale Models of Visual Motion Perception" facets, sciblog
stripping title : --------------
stripping title : 
stripping title : Deliverable M9-3: Workshop for definition of a detailed version of the V1 hypercolumn model
stripping title : ===========================================================================================
Warning     PDF <https://invibe.net/LaurentPerrinet/SciBlog/2007-10-22?action=AttachFile&do=view&target=Meeting_V1WorkshopMarseille_2007.pdf>`__
Warning  `Marseille\_November2006 <https://invibe.net/LaurentPerrinet/SciBlog/2006-11-20>`__ ).
07-10-22 Deliverable M9-3: Workshop for definition of a detailed version of the V1 hypercolumn model facets, sciblog
stripping title : keyboard
stripping title : ========
08-08-29 keyboard macos, sciblog
stripping title : --------------
stripping title : 
stripping title : Impact Factor
stripping title : =============
Warning     paper <https://invibe.net/LaurentPerrinet/SciBlog/2008-11-20?action=AttachFile&do=get&target=TaylorPerakakisTrachana2008.pdf>`__
08-11-20 Impact Factor sciblog
stripping title : --------------
stripping title : 
stripping title : duplicate files
stripping title : ===============
08-12-05 duplicate files info, sciblog
stripping title : some LaTeX tips: counting words / install on macosx
stripping title : ===================================================
09-08-19 Installation TeX latex
stripping title : some LaTeX tips: drafts, links, margins, pdflatex
stripping title : =================================================
09-08-20 some LaTeX tips: drafts, links, margins, pdflatex latex
stripping title : setting graphics' path
stripping title : ======================
No old file 09-08-21-setting-graphics'-path.rst
09-08-21 setting graphics' path latex
stripping title : List Of Symbols
stripping title : ===============
09-08-23 List Of Symbols latex
stripping title : Creating Proceedings (almost) automatically using python and latex
stripping title : ==================================================================
Warning  `Makefile <https://invibe.net/LaurentPerrinet/SciBlog/2009-08-26?action=AttachFile&do=view&target=Makefile>`__
Warning     `body.py <https://invibe.net/LaurentPerrinet/SciBlog/2009-08-26?action=AttachFile&do=view&target=body.py>`__
Warning     `body.tex <https://invibe.net/LaurentPerrinet/SciBlog/2009-08-26?action=AttachFile&do=view&target=body.tex>`__
Warning     `neurocomp08proceedings.tex <https://invibe.net/LaurentPerrinet/SciBlog/2009-08-26?action=AttachFile&do=view&target=neurocomp08proceedings.tex>`__:
Warning        `Makefile <https://invibe.net/LaurentPerrinet/SciBlog/2009-08-26?action=AttachFile&do=view&target=Makefile>`__
Warning        `titlepage.tex <https://invibe.net/LaurentPerrinet/SciBlog/2009-08-26?action=AttachFile&do=view&target=titlepage.tex>`__:
Warning        `body.tex <https://invibe.net/LaurentPerrinet/SciBlog/2009-08-26?action=AttachFile&do=view&target=body.tex>`__
Warning     `Makefile <https://invibe.net/LaurentPerrinet/SciBlog/2009-08-26?action=AttachFile&do=view&target=Makefile>`__
Warning     `voilà! <https://invibe.net/LaurentPerrinet/SciBlog/2009-08-26?action=AttachFile&do=view&target=neurocomp08proceedings.pdf>`__
09-08-26 Creating Proceedings (almost) automatically using python and latex latex
stripping title : Tips on Filesystems, security and al on mac os x
stripping title : ================================================
09-08-29 Tips on Filesystems, security and al on mac os x macos, sciblog
stripping title : --------------
stripping title : 
stripping title : floating point magic
stripping title : ====================
09-11-01 floating point magic sciblog
stripping title : --------------
stripping title : 
stripping title : MoinMoin: howto install a new theme
stripping title : ===================================
09-11-15 MoinMoin: howto install a new theme sciblog
stripping title : --------------
stripping title : 
stripping title : some unix tips
stripping title : ==============
09-11-22 some unix tips sciblog
stripping title : --------------
stripping title : 
stripping title : inkscape native
stripping title : ===============
09-11-26 inkscape native macos, sciblog
stripping title : --------------
stripping title : 
stripping title : CRAAC: Compte rendu annuel d'activité des chercheurs du CNRS. Année 2008 - 2009
stripping title : ===============================================================================
No old file 09-12-02-CRAAC-Compte-rendu-annuel-d'activité-des-chercheurs-du-CNRS.-Année-2008-2009.rst
09-12-02 CRAAC: Compte rendu annuel d'activité des chercheurs du CNRS. Année 2008 - 2009 sciblog
stripping title : --------------
stripping title : 
stripping title : convert a bitmap image to a vectorized PDF using mkbitmap and potrace
stripping title : =====================================================================
09-12-06 convert a bitmap image to a vectorized PDF using mkbitmap and potrace sciblog
stripping title : --------------
stripping title : 
stripping title : Journal Club : The good, the bad and the NOISE
stripping title : ==============================================
Warning     (63Mo) <https://invibe.net/LaurentPerrinet/SciBlog/2009-12-09?action=AttachFile&do=get&target=09-12-10_journal_club_noise.pdf>`__
09-12-09 Journal Club : The good, the bad and the NOISE motionclouds, sciblog
stripping title : --------------
stripping title : 
stripping title : which or that?
stripping title : ==============
09-12-11 which or that? sciblog
stripping title : --------------
stripping title : 
stripping title : Assistance Mail Free
stripping title : ====================
09-12-17 Assistance Mail Free sciblog
stripping title : contributing to the python community
stripping title : ====================================
09-12-19 contributing to the python community sciblog
stripping title : --------------
stripping title : 
stripping title : The original eve
stripping title : ================
10-01-27 The original eve sciblog
stripping title : --------------
stripping title : 
stripping title : quelques blogs français de science
stripping title : ==================================
10-02-03 quelques blogs français de science sciblog
stripping title : appel d'offres 2010 "Neuroinformatique et neurosciences computationnelles"
stripping title : ==========================================================================
10-02-11 Neuroinformatique et neurosciences computationnelles computationalneuroscience
stripping title : Using Ssh, the Secure Shell
stripping title : ===========================
10-03-15 securing the server macos, sciblog
stripping title : --------------
stripping title : 
stripping title : using grin
stripping title : ==========
10-04-07 using grin sciblog
stripping title : Comment créer et manipuler les données scientifiques : autour de Numpy
stripping title : ======================================================================
No old file 10-04-08-Comment-créer-et-manipuler-les-données-scientifiques-autour-de-Numpy.rst
10-04-08 Comment créer et manipuler les données scientifiques : autour de Numpy sciblog
stripping title : --------------
stripping title : 
stripping title : Richard Dawkins on our "queer" universe
stripping title : =======================================
10-04-24 Richard Dawkins on our "queer" universe sciblog
stripping title : --------------
stripping title : 
stripping title : bibdesk + citeulike
stripping title : ===================
10-04-25 bibdesk + citeulike bibcloud, sciblog
stripping title : replacing text in files
stripping title : =======================
10-04-26 replacing text in files sciblog
stripping title : --------------
stripping title : 
stripping title : reStructuredText rst cheatsheet
stripping title : ===============================
10-04-28 reStructuredText rst cheatsheet sciblog
stripping title : --------------
stripping title : 
stripping title : sanitize a bibtex file using bibtool
stripping title : ====================================
10-05-22 sanitize a bibtex file using bibtool bibcloud, sciblog
stripping title : Haïm Cohen : Tu Ne Laisseras Point Pleurer
stripping title : ==========================================
10-05-23 Haïm Cohen : Tu Ne Laisseras Point Pleurer sciblog
stripping title : --------------
stripping title : 
stripping title : latex within moin
stripping title : =================
10-07-08 latex within moin moin, sciblog, latex
stripping title : compiling OpenCV on MacOSX 10.6
stripping title : ===============================
No old file 10-08-03-compiling-OpenCV-on-MacOSX-10.6.rst
10-08-03 compiling OpenCV on MacOSX 10.6 macos, sciblog
stripping title : --------------
stripping title : 
stripping title : Pinna illusion
stripping title : ==============
10-08-05 Pinna illusion sciblog
stripping title : --------------
stripping title : 
stripping title : distributed computing
stripping title : =====================
10-08-07 distributed computing sciblog
stripping title : running embarassingly parallel simulations on a multicore machine using bash loops
stripping title : ==================================================================================
10-08-09 running embarassingly parallel simulations on a multicore machine using bash loops sciblog
stripping title : --------------
stripping title : 
stripping title : getting the PID from matlab
stripping title : ===========================
10-08-11 getting the PID from matlab sciblog
stripping title : System Updates
stripping title : ==============
10-08-29 System Updates macos, sciblog
stripping title : Terminal.app shortcuts
stripping title : ======================
No old file 10-08-30-Terminal.app-shortcuts.rst
10-08-30 Terminal.app shortcuts macos, sciblog
stripping title : managing packages on MacOsX : testing HomeBrew
stripping title : ==============================================
10-08-31 managing packages on MacOsX : testing HomeBrew macos, sciblog
stripping title : A neurocentric approach to Bayesian inference
stripping title : =============================================
10-09-03 A neurocentric approach to Bayesian inference sciblog
stripping title : --------------
stripping title : 
stripping title : Journée de l'IFR 131 -Sciences du Cerveau et de la Cognition
stripping title : ============================================================
Warning     présentation <https://invibe.net/LaurentPerrinet/SciBlog/2010-09-09?action=AttachFile&do=view&target=10-09-09_journee_IFR_noise.pdf>`__
No old file 10-09-09-Journée-de-l'IFR-131-Sciences-du-Cerveau-et-de-la-Cognition.rst
10-09-09 Journée de l'IFR 131 -Sciences du Cerveau et de la Cognition sciblog
stripping title : --------------
stripping title : 
stripping title : bundling using py2app
stripping title : =====================
10-09-27 bundling using py2app macos, sciblog
stripping title : --------------
stripping title : 
stripping title : gmail Exchange ActiveSync
stripping title : =========================
10-10-05 gmail Exchange ActiveSync sciblog
stripping title : --------------
stripping title : 
stripping title : Émergence
stripping title : =========
10-10-06 Émergence sciblog
stripping title : --------------
stripping title : 
stripping title : nous
stripping title : ====
10-10-09 nous sciblog
stripping title : installing python and its components
stripping title : ====================================
Warning     `TagSciBlog <https://invibe.net/LaurentPerrinet/TagSciBlog>`__
10-10-27 installing python and its components macos, sciblog
stripping title : --------------
stripping title : 
stripping title : Marseille : bookmarks and tips
stripping title : ==============================
10-10-31 Marseille : bookmarks and tips sciblog
stripping title : --------------
stripping title : 
stripping title : rsync to an alternate ssh port
stripping title : ==============================
10-11-03 rsync to an alternate ssh port sciblog
stripping title : --------------
stripping title : 
stripping title : installing Dovecot on MacOsX using MacPorts
stripping title : ===========================================
10-11-04 installing Dovecot on MacOsX using MacPorts using, macos, sciblog
stripping title : --------------
stripping title : 
stripping title : wma to MP3
stripping title : ==========
10-11-07 wma to MP3 sciblog
stripping title : --------------
stripping title : 
stripping title : Instinct Paradise : journée IMERA du 9 nov 2010
stripping title : ===============================================
No old file 10-11-09-Instinct-Paradise-journée-IMERA-du-9-nov-2010.rst
10-11-09 Instinct Paradise :-journée IMERA du 9 nov 2010 tropique, sciblog
stripping title : --------------
stripping title : 
stripping title : Password-less logins with OpenSSH
stripping title : =================================
10-11-15 Password-less logins with OpenSSH sciblog
stripping title : --------------
stripping title : 
stripping title : ignoring a folder in SVN
stripping title : ========================
10-11-27 ignoring a folder in SVN sciblog
stripping title : --------------
stripping title : 
stripping title : CRAAC: Compte rendu annuel d'activité des chercheurs du CNRS. Année 2010
stripping title : ========================================================================
No old file 10-12-08-CRAAC-Compte-rendu-annuel-d'activité-des-chercheurs-du-CNRS.-Année-2010.rst
10-12-08 CRAAC: Compte rendu annuel d'activité des chercheurs du CNRS. Année 2010 sciblog
stripping title : --------------
stripping title : 
stripping title : Caps Lock, what a useless key
stripping title : =============================
10-12-20 Caps Lock, what a useless key macos, sciblog
stripping title : how to find stuff
stripping title : =================
10-12-27 how to find stuff info, sciblog
stripping title : --------------
stripping title : 
stripping title : ubuntu : starting sshd at boot
stripping title : ==============================
11-01-18 ubuntu : starting sshd at boot sciblog
stripping title : --------------
stripping title : 
stripping title : inverting colors in MacOsX
stripping title : ==========================
11-01-19 inverting colors in MacOsX macos, sciblog
stripping title : --------------
stripping title : 
stripping title : Finder Shortcuts in MacOsX
stripping title : ==========================
11-01-25 Finder Shortcuts in MacOsX macos, sciblog
stripping title : --------------
stripping title : 
stripping title : python in user space
stripping title : ====================
11-01-26 python in user space sciblog
stripping title : --------------
stripping title : 
stripping title : handling processes in bash
stripping title : ==========================
11-01-29 handling processes in bash sciblog
stripping title : --------------
stripping title : 
stripping title : pmset: selecting the sleep mode in Mac Os X
stripping title : ===========================================
11-02-09 pmset: selecting the sleep mode in Mac Os X sciblog
stripping title : --------------
stripping title : 
stripping title : 10.10 64bit AHCI hosed on a dell T3500
stripping title : ======================================
No old file 11-02-21-10.10-64bit-AHCI-hosed-on-a-dell-T3500.rst
11-02-21 10.10 64bit AHCI hosed on a dell T3500 sciblog
stripping title : --------------
stripping title : 
stripping title : Change User ID and Group ID in Snow Leopard
stripping title : ===========================================
11-02-22 Change User ID and Group ID in Snow Leopard macos, sciblog
stripping title : --------------
stripping title : 
stripping title : Ermentrout : "Double or Nothing: Phosphenes and the periodic driving of cortex"
stripping title : ===============================================================================
11-03-09 Ermentrout : "Double or Nothing: Phosphenes and the periodic driving of cortex" sciblog
stripping title : --------------
stripping title : 
stripping title : SpikeStream & Nemo
stripping title : ==================
11-03-10 SpikeStream & Nemo sciblog
stripping title : --------------
stripping title : 
stripping title : HomeBrew: compiling a python toolchain
stripping title : ======================================
11-05-04 HomeBrew: compiling a python toolchain sciblog
stripping title : Publications 2006-2010
stripping title : ======================
Warning  #. `SciBlog/2011-07-05 <https://invibe.net/LaurentPerrinet/SciBlog/2011-07-05?highlight=%28TagPublicationsArticles%29%7C%28TagYear06%29%7C%28TagYear07%29%7C%28TagYear08%29%7C%28TagYear09%29%7C%28TagYear10%29>`__
11-07-05 Publications 2006-2010 publications2006-2009, year09, year08, publications2006-2010, year10), sciblog, year07
stripping title : --------------
stripping title : 
stripping title : scripting MoinMoin to get, change or rename pages
stripping title : =================================================
11-07-06 scripting MoinMoin to get, change or rename pages moin, sciblog
stripping title : using a versioning system
stripping title : =========================
11-07-07 using a versioning system macos, sciblog
stripping title : --------------
stripping title : 
stripping title : mercurial & LaTeX
stripping title : =================
11-07-08 mercurial & LaTeX latex, sciblog
stripping title : --------------
stripping title : 
stripping title : mounting filesystems using SSH
stripping title : ==============================
11-07-10 mounting filesystems using SSH info, macos, sciblog
stripping title : --------------
stripping title : 
stripping title : computational and theoretical neuroscience
stripping title : ==========================================
11-07-25 computational and theoretical neuroscience sciblog, computationalneuroscience
stripping title : --------------
stripping title : 
stripping title : installing MoinMoin on a OVH server
stripping title : ===================================
12-01-07 installing MoinMoin on a OVH server moin, sciblog
stripping title : --------------
stripping title : 
stripping title : Compiling pyglet on MacOsX
stripping title : ==========================
12-01-18 Compiling pyglet on MacOsX macos, sciblog
stripping title : --------------
stripping title : 
stripping title : installing Dovecot on debian
stripping title : ============================
12-02-08 installing Dovecot on debian using, sciblog
stripping title : --------------
stripping title : 
stripping title : Tropique : intervention Enghien
stripping title : ===============================
12-02-16 Tropique :-intervention Enghien year12, tropique, sciblog
stripping title : --------------
stripping title : 
stripping title : paramétrer l'e-mail à l'INT
stripping title : ===========================
No old file 12-02-17-paramétrer-l'e-mail-à-l'INT.rst
12-02-17 paramétrer l'e-mail à l'INT int, sciblog
stripping title : --------------
stripping title : 
stripping title : access spawn
stripping title : ============
12-02-22 access spawn int, sciblog
stripping title : --------------
stripping title : 
stripping title : Completely disable quarantine of downloaded files
stripping title : =================================================
12-03-05 Completely disable quarantine of downloaded files macos, sciblog
stripping title : --------------
stripping title : 
stripping title : view pdf online on firefox in macosx
stripping title : ====================================
12-03-10 view pdf online on firefox in macosx macos, sciblog
stripping title : --------------
stripping title : 
stripping title : Stop Bonjour from broadcasting ssh and sftp
stripping title : ===========================================
12-03-11 Stop Bonjour from broadcasting ssh and sftp int, macos, sciblog
stripping title : --------------
stripping title : 
stripping title : Doing a red-lined article file from two versions of a paper in LaTeX
stripping title : ====================================================================
12-03-12 Doing a red-lined article file from two versions of a paper in LaTeX latex, sciblog
stripping title : --------------
stripping title : 
stripping title : Moving Time Machine to a New Hard Drive
stripping title : =======================================
12-03-13 Moving Time Machine to a New Hard Drive macos, sciblog
stripping title : --------------
stripping title : 
stripping title : Creating a bootable Debian USB flash drive on MacOsX
stripping title : ====================================================
12-03-16 Creating a bootable Debian USB flash drive on MacOsX info, macos, sciblog
stripping title : --------------
stripping title : 
stripping title : installing Dovecot on MacOsX using HomeBrew
stripping title : ===========================================
12-03-21 installing Dovecot on MacOsX using HomeBrew using, macos, sciblog
stripping title : --------------
stripping title : 
stripping title : how to leave iphoto
stripping title : ===================
12-03-22 how to leave iphoto macos, sciblog
stripping title : --------------
stripping title : 
stripping title : Type your paths directly
stripping title : ========================
12-03-24 Type your paths directly macos, sciblog
stripping title : --------------
stripping title : 
stripping title : going back to simplicity: duckduckgo
stripping title : ====================================
12-03-25 going back to simplicity: duckduckgo sciblog
stripping title : --------------
stripping title : 
stripping title : managing defaults on MacOsX
stripping title : ===========================
12-04-05 managing defaults on MacOsX sciblog
stripping title : --------------
stripping title : 
stripping title : transition from movie15.sty to media9.sty
stripping title : =========================================
No old file 12-04-17-transition-from-movie15.sty-to-media9.sty.rst
12-04-17 transition from movie15.sty to media9.sty latex, sciblog
stripping title : --------------
stripping title : 
stripping title : Converting FLAC to AAC (or MP3 to OGG etc...)
stripping title : =============================================
No old file 12-04-22-Converting-FLAC-to-AAC-(or-MP3-to-OGG-etc...).rst
12-04-22 Converting FLAC to AAC (or MP3 to OGG etc...) sciblog
stripping title : --------------
stripping title : 
stripping title : rotating a video using ffmpeg
stripping title : =============================
12-04-30 rotating a video using ffmpeg sciblog
stripping title : --------------
stripping title : 
stripping title : racourci pour accéder à une page INIST
stripping title : ======================================
No old file 12-06-12-racourci-pour-accéder-à-une-page-INIST.rst
12-06-12 racourci pour accéder à une page INIST int, sciblog
stripping title : --------------
stripping title : 
stripping title : using and re-using metadata in LaTeX
stripping title : ====================================
12-07-10 using and re-using metadata in LaTeX latex, sciblog
stripping title : --------------
stripping title : 
stripping title : Make PDF files searchable and copyable
stripping title : ======================================
12-07-24 Make PDF files searchable and copyable latex, sciblog
stripping title : --------------
stripping title : 
stripping title : István Orosz
stripping title : ============
No old file 12-08-19-István-Orosz.rst
12-08-19 István Orosz sciblog
stripping title : --------------
stripping title : 
stripping title : installing Dovecot on QNAP
stripping title : ==========================
12-08-27 installing Dovecot on QNAP using, sciblog
stripping title : --------------
stripping title : 
stripping title : connecting a linux client to a QNAP's LDAP server
stripping title : =================================================
No old file 12-09-04-connecting-a-linux-client-to-a-QNAP's-LDAP-server.rst
12-09-04 connecting a linux client to a QNAP's LDAP server info, sciblog
stripping title : --------------
stripping title : 
stripping title : installing linux on imac hardware
stripping title : =================================
12-09-09 installing linux on imac hardware sciblog
stripping title : |/!\\| **not public**
stripping title : 
stripping title : --------------
stripping title : 
stripping title : discover ports on MacOSX
stripping title : ========================
12-09-30 discover ports on MacOSX macos, sciblog
stripping title : --------------
stripping title : 
stripping title : craac 2012
stripping title : ==========
12-11-07 craac 2012 sciblog
stripping title : --------------
stripping title : 
stripping title : setting the umask to define default permissions for created files
stripping title : =================================================================
12-11-28 setting the umask to define default permissions for created files int, sciblog
stripping title : --------------
stripping title : 
stripping title : WP4 report : NeuroTools support for the synthesis of random textured dynamical stimuli
stripping title : ======================================================================================
13-01-16 WP4 report : NeuroTools support for the synthesis of random textured dynamical stimuli sciblog, brainscales
stripping title : --------------
stripping title : 
stripping title : (re)moving lots of file containing a similar pattern
stripping title : ====================================================
13-01-31 (re)moving lots of file containing a similar pattern sciblog
stripping title : --------------
stripping title : 
stripping title : How To Change Your Time Machine Backup Interval
stripping title : ===============================================
13-02-02 How To Change Your Time Machine Backup Interval macos, sciblog
stripping title : --------------
stripping title : 
stripping title : WP5 Year 2 report: contribution of CNRS-INT (Institut de Neurosciences de la Timone)
stripping title : ====================================================================================
13-02-04 WP5 Year 2 report: contribution of CNRS-INT (Institut de Neurosciences de la Timone) sciblog, brainscales
stripping title : --------------
stripping title : 
stripping title : setting some conventions common in Bibdesk (to work with bibtex, citeUlike)
stripping title : ===========================================================================
13-03-06 setting some conventions common in Bibdesk (to work with bibtex, citeUlike) sciblog
stripping title : --------------
stripping title : 
stripping title : shortcut to put the display to sleep
stripping title : ====================================
13-03-13 shortcut to put the display to sleep macos, sciblog
stripping title : --------------
stripping title : 
stripping title : upgrading owncloud to 5.0.0
stripping title : ===========================
No old file 13-03-23-upgrading-owncloud-to-5.0.0.rst
13-03-23 upgrading owncloud to 5.0.0 sciblog
stripping title : --------------
stripping title : 
stripping title : using tabs in vim
stripping title : =================
13-03-25 using tabs in vim info, sciblog
stripping title : --------------
stripping title : 
stripping title : dropping owncloud
stripping title : =================
13-05-14 dropping owncloud sciblog
stripping title : --------------
stripping title : 
stripping title : updating to mactex (texlive for mac), version 2013
stripping title : ==================================================
13-06-12 updating to mactex (texlive for mac), version 2013 latex, sciblog
stripping title : --------------
stripping title : 
stripping title : Useful numbers
stripping title : ==============
13-07-08 Useful numbers sciblog
stripping title : --------------
stripping title : 
stripping title : Meshnet
stripping title : =======
13-10-26 Meshnet sciblog
stripping title : --------------
stripping title : 
stripping title : !SparkleShare on a QNAP
stripping title : =======================
13-11-04 !SparkleShare on a QNAP using, sciblog
stripping title : --------------
stripping title : 
stripping title : homebrew cask (level 0): contributing a new cask
stripping title : ================================================
Warning     `mistyped <https://invibe.net/LaurentPerrinet/SciBlog/2013-11-08>`__ something...
13-11-08 homebrew cask (level 0): contributing a new cask macos, sciblog
stripping title : --------------
stripping title : 
stripping title : homebrew cask (level 1): correcting a pull request
stripping title : ==================================================
Warning  -  I mistyped `my contribution <https://invibe.net/LaurentPerrinet/SciBlog/2013-11-08>`__,
13-11-09 homebrew cask (level 1): correcting a pull request macos, sciblog
stripping title : --------------
stripping title : 
stripping title : How Airplanes Fly: A Physical Description of Lift
stripping title : =================================================
14-01-01 How Airplanes Fly: A Physical Description of Lift sciblog

In [104]:

todo_page_list = """\
SciBlog/2012-12-19
SciBlog/2012-12-12
SciBlog/2011-07-18
SciBlog/2011-07-12
SciBlog/2012-10-02
"""

clean-up of failed pages

In [122]:

import os
#print os.getcwd()
for page in todo_page_list.split('\n'):
    DATE = page.replace('SciBlog/', '')
    if len(DATE)>2:
        cmd = 'rm -fr %s*' % DATE[2:]
        print cmd
        os.system(cmd)

rm -fr 12-12-19*
rm -fr 12-12-12*
rm -fr 11-07-18*
rm -fr 11-07-12*
rm -fr 12-10-02*

extracting title and tags¶

Done!¶