Reproducing Olshausen's classical SparseNet (part 3)

Laurent Perrinet

2017-03-16 13:51

In this notebook, we test the convergence of SparseNet as a function of different learning parameters. This shows the relative robustness of this method according to the coding parameters, but also the importance of homeostasis to obtain an efficient set of filters:

first, whatever the learning rate, the convergence is not complete without homeostasis,
second, we achieve better convergence for similar learning rates and on a certain range of learning rates for the homeostasis
third, the smoothing parameter alpha_homeo has to be properly set to achieve a good convergence.
last, this homeostatic rule works with the different variants of sparse coding.

1. With diferent learning rates but without homeostasis¶

Here,we only ensure the norm ofthe filters is constant.

In [3]:

shl = SHL()
data = shl.get_data()

for eta in np.logspace(-1, 1, N_scan, base=10)*shl.eta: matname = 'no homeo - eta={}'.format(eta) shl = SHL(DEBUG_DOWNSCALE=DEBUG_DOWNSCALE, eta_homeo=0, eta=eta, verbose=verbose) dico = shl.learn_dico(data=data, matname=matname, list_figures=list_figures)

2. Homeostasis à-la-SparseNet¶

In [4]:

shl = SHL()
data = shl.get_data()
for eta in np.logspace(-1, 1, N_scan, base=10)*shl.eta:
    matname = 'homeo - eta={}'.format(eta)
    shl = SHL(DEBUG_DOWNSCALE=DEBUG_DOWNSCALE, eta=eta, verbose=verbose)
    dico = shl.learn_dico(data=data, matname=matname, list_figures=list_figures)

No description has been provided for this image

In [5]:

dico.record_each

Out[5]:

In [6]:

shl = SHL()
data = shl.get_data()
for eta_homeo in np.logspace(-1, 1, N_scan, base=10)*shl.eta_homeo:
    matname = 'homeo - eta_homeo={}'.format(eta_homeo)
    shl = SHL(DEBUG_DOWNSCALE=DEBUG_DOWNSCALE, eta_homeo=eta_homeo, verbose=verbose)
    dico = shl.learn_dico(data=data, matname=matname, list_figures=list_figures)

3. with different smoothing parameters for the homestatic gain¶

In [7]:

shl = SHL()
data = shl.get_data()
for alpha_homeo in np.logspace(-1, .5, N_scan, base=10)*shl.alpha_homeo:
    matname = 'homeo - alpha_homeo={}'.format(alpha_homeo)

    print(alpha_homeo)
    shl = SHL(DEBUG_DOWNSCALE=DEBUG_DOWNSCALE, alpha_homeo=alpha_homeo, verbose=verbose)
    dico = shl.learn_dico(data=data, matname=matname, list_figures=list_figures)

0.02
0.0355655882008
0.0632455532034
0.112468265038
0.2
0.355655882008
0.632455532034

4. with different sparse parameters¶

In [8]:

N_scan = int(15/(DEBUG_DOWNSCALE)**.3)
for l0_sparseness in 2**np.arange(8):
    matname = 'l0_sparseness={}'.format(l0_sparseness)
    shl = SHL(DEBUG_DOWNSCALE=DEBUG_DOWNSCALE, l0_sparseness=l0_sparseness, verbose=verbose)
    dico = shl.learn_dico(data=data, matname=matname, list_figures=list_figures)

5. with different sparse coding algorithms¶

In [9]:

learning_algorithms = [
    ('Matching Pursuit 10 atom', 'MP_N10',
     {'learning_algorithm':'mp', 'l0_sparseness': 10}),
    ('Matching Pursuit 20 atom', 'MP_N20',
     {'learning_algorithm':'mp', 'l0_sparseness': 20}),
    ('Orthogonal Matching Pursuit 10 atom', 'OMP_N10',
     {'learning_algorithm':'omp', 'l0_sparseness': 10}),
    ('Orthogonal Matching Pursuit 20 atom', 'OMP_N20',
     {'learning_algorithm':'omp', 'l0_sparseness': 20}),
    ('Least-angle regression 5 atoms', 'LARS',
     {'learning_algorithm':'lars', 'l0_sparseness': 5}),
    ('Least-angle regression 5 atoms', 'LARS',
     {'learning_algorithm':'lasso_lars', 'l0_sparseness': 5}),
    ('Least-angle regression 5 atoms', 'LARS',
     {'learning_algorithm':'lasso_cd', 'l0_sparseness': 5})]
        
for learning_title, learning_label, learning_kwargs in learning_algorithms:
    print('Dictionary learned from image patches using ' + learning_title)
    shl = SHL(DEBUG_DOWNSCALE=DEBUG_DOWNSCALE, database=database,
              verbose=verbose, **learning_kwargs)
    dico = shl.learn_dico(data=data, matname=learning_label, list_figures=list_figures)

Dictionary learned from image patches using Matching Pursuit 10 atom
Dictionary learned from image patches using Matching Pursuit 20 atom
Dictionary learned from image patches using Orthogonal Matching Pursuit 10 atom

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-9-58c588e2b225> in <module>()
     19     shl = SHL(DEBUG_DOWNSCALE=DEBUG_DOWNSCALE, database=database,
     20               verbose=verbose, **learning_kwargs)
---> 21     dico = shl.learn_dico(data=data, matname=learning_label, list_figures=list_figures)
     22 

/Users/laurentperrinet/pool/science/BICV/SHL_scripts/shl_scripts/shl_scripts.py in learn_dico(self, data, name_database, matname, list_figures, **kwargs)
    175                 else:
    176                     dico = 'lock'
--> 177                     print('the computation is locked', fmatname + LOCK)
    178             else:
    179                 with open(fmatname, 'rb') as fp:

NameError: name 'LOCK' is not defined

Version used¶

In [ ]:

import version_information
%version_information numpy, shl_scripts