<commit_before>"""

=========================
Reconstruct with a Tensor
=========================

This example shows how to reconstruct your diffusion datasets using a single Tensor model.

First import the necessary modules:

``numpy`` is for numerical computation

"""

import numpy as np

"""
``nibabel`` is for loading imaging datasets
"""

import nibabel as nib

"""
``dipy.reconst`` is for the reconstruction algorithms which we use to create voxel models from the raw data.
"""

from dipy.reconst.dti import TensorModel

"""
``dipy.data`` is used for small datasets that we use in tests and examples.
"""

from dipy.data import fetch_stanford_hardi

"""
Fetch will download the raw dMRI dataset of a single subject. The size of the dataset is 51 MBytes.
You only need to fetch once.
"""

fetch_stanford_hardi()

"""
Next, we read the saved dataset
"""

from dipy.data import read_stanford_hardi

img, gtab = read_stanford_hardi()

"""
img contains a nibabel Nifti1Image object (with the data) and gtab contains a GradientTable
object (information about the gradients e.g. b-values and b-vectors).
"""

data = img.get_data()
print('data.shape (%d, %d, %d, %d)' % data.shape)

"""
data.shape ``(128, 128, 49, 65)``

This dataset has anisotropic voxel sizes, therefore reslicing is necessary
"""

affine = img.get_affine()

"""
Load and show the zooms which hold the voxel size.
"""

zooms = img.get_header().get_zooms()[:3]

"""
The voxel size here is ``(1.79, 1.79, 2.5)``.

We now set the required new voxel size.
"""

new_zooms = (2., 2., 2.)

"""
Start reslicing. Trilinear interpolation is used by default.
"""

from dipy.align.aniso2iso import resample

data2, affine2 = resample(data, affine, zooms, new_zooms)

print('data2.shape (%d, %d, %d, %d)' % data2.shape)

"""
data2.shape ``(115, 115, 61, 65)``

Lets also create a simple mask. This is a quick way to avoid calculating
Tensors on the background of the image. This is because the signal is very low in
these region. A better way would be to extract the brain region using a brain
extraction method. But will skip that for now.
"""

mask = data2[..., 0] > 50

"""
Now that we have prepared the datasets we can go forward with the voxel
reconstruction. First, we instantiate the Tensor model in the following way.
"""

tenmodel = TensorModel(gtab)

"""
Fitting the data is very simple. We just need to call the fit method of the
TensorModel in the following way:
"""

tenfit = tenmodel.fit(data2, mask)

"""
The fit method creates a TensorFit object which contains the fitting parameters
and other attributes of the model. For example we can generate fractional
anisotropy from the eigen values of the single tensor.
"""

from dipy.reconst.dti import fractional_anisotropy

FA = fractional_anisotropy(tenfit.evals)

"""
In the background of the image the fitting will not be accurate there is no
signal and possibly we will find FA values with nans (not a number). We can
easily remove these in the following way.
"""

FA[np.isnan(FA)] = 0

"""
Saving the FA images is very easy using nibabel. We need the FA volume and the
affine matrix which transform the image's coordinates to the world coordinates.
"""

fa_img = nib.Nifti1Image(FA, img.get_affine())
nib.save(fa_img, 'tensor_fa.nii.gz')

"""
You can now see the result with any nifti viewer or check it slice by slice
using matplotlib_'s imshow. In the same way you can save the eigen values the
eigen vectors or any other properties of the Tensor.
"""

evecs_img = nib.Nifti1Image(tenfit.evecs, img.get_affine())
nib.save(evecs_img, 'tensor_evecs.nii.gz')

"""
Finally lets try to visualize the orientation distribution functions of a small
rectangular area around the middle of our datasets.
"""

i,j,k,w = np.array(data2.shape) / 2
data_small  = data2[i-5:i+5, j-5:j+5, k-2:k+2]
from dipy.data import get_sphere
sphere = get_sphere('symmetric724')

from dipy.viz import fvtk
r = fvtk.ren()
fvtk.add(r, fvtk.sphere_funcs(tenmodel.fit(data_small).odf(sphere),
							  sphere, colormap=None))

print('Saving illustration as tensor_odfs.png')
fvtk.record(r, n_frames=1, out_path='tensor_odfs.png', size=(600, 600))

"""
.. figure:: tensor_odfs.png
   :align: center

   **Tensor ODFs**.

.. include:: ../links_names.inc

"""
<commit_msg>DOC: Added some meat to the tensor example.<commit_after>"""

============================================================
Reconstruction of the diffusion signal with the tensor model
============================================================

The diffusion tensor model is a phenomenological model of the diffusion within
a voxel. First proposed by Basser and colleagues _[Basser1994], it has been
extremely influential in demonstrating the utility of diffusion MRI in
characterizing the micro-structure of white matter tissue and of the
biophysical properties of tissue, inferred from local diffusion properties.

The diffusion tensor models the diffusion signal as:

.. math::

    \frac{S(\theta, b)}{S_0} = e^{-b\theta^t Q \theta}

Where $\theta$ is a unit vector in 3 space indicating the direction of
measurement and b are the parameters of measurement, such as the strength and
duration of diffusion-weighting gradient. $S(\theta, b)$ is the
diffusion-weighted signal measured and $S_0$ is the signal conducted in a
measurement with no diffusion weighting. Q is a positive-definite quadratic
form, which contains six free parameters to be fit. These six parameters are:

.. math::

   Q = \begin{pmatrix} D_{xx} & D_{xy} & D_{xz} \\
                       D_{yx} & D_{yy} & D_{yz} \\
                       D_{zx} & D_{zy} & D_{zz} \\ \end{pmatrix}

This matrix is a variance/covariance matrix of the diffusivity along the three
spatial dimensions. Note that we can assume that diffusivity has antipodal
symmetry, so elements across the diagonal are equal. For example:
$D_{xy} = D_{yx}$. This is why there are only 6 free parameters to estimate
here.

In the following example we show how to reconstruct your diffusion datasets
using a single tensor model.

First import the necessary modules:

``numpy`` is for numerical computation

"""

import numpy as np

"""
``nibabel`` is for loading imaging datasets
"""

import nibabel as nib

"""
``dipy.reconst`` is for the reconstruction algorithms which we use to create
voxel models from the raw data.
"""

from dipy.reconst.dti import TensorModel

"""
``dipy.data`` is used for small datasets that we use in tests and examples.
"""

from dipy.data import fetch_stanford_hardi

"""
Fetch will download the raw dMRI dataset of a single subject. The size of the
dataset is 51 MBytes.  You only need to fetch once.
"""

fetch_stanford_hardi()

"""
Next, we read the saved dataset
"""

from dipy.data import read_stanford_hardi

img, gtab = read_stanford_hardi()

"""
img contains a nibabel Nifti1Image object (with the data) and gtab contains a
GradientTable object (information about the gradients e.g. b-values and
b-vectors).
"""

data = img.get_data()
print('data.shape (%d, %d, %d, %d)' % data.shape)

"""
data.shape ``(128, 128, 49, 65)``

This dataset has anisotropic voxel sizes, therefore reslicing is necessary
"""

affine = img.get_affine()

"""
Load and show the zooms which hold the voxel size.
"""

zooms = img.get_header().get_zooms()[:3]

"""
The voxel size here is ``(1.79, 1.79, 2.5)``.

We now set the required new voxel size.
"""

new_zooms = (2., 2., 2.)

"""
Start reslicing. Trilinear interpolation is used by default.
"""

from dipy.align.aniso2iso import resample

data2, affine2 = resample(data, affine, zooms, new_zooms)

print('data2.shape (%d, %d, %d, %d)' % data2.shape)

"""
data2.shape ``(115, 115, 61, 65)``

Lets also create a simple mask. This is a quick way to avoid calculating
Tensors on the background of the image. This is because the signal is very low in
these region. A better way would be to extract the brain region using a brain
extraction method. But will skip that for now.
"""

mask = data2[..., 0] > 50

"""
Now that we have prepared the datasets we can go forward with the voxel
reconstruction. First, we instantiate the Tensor model in the following way.
"""

tenmodel = TensorModel(gtab)

"""
Fitting the data is very simple. We just need to call the fit method of the
TensorModel in the following way:
"""

tenfit = tenmodel.fit(data2, mask)

"""
The fit method creates a TensorFit object which contains the fitting parameters
and other attributes of the model. For example we can generate fractional
anisotropy (FA) from the eigen-values of the tensor. FA is used to characterize
the degree to which the distribution of diffusion in a voxel is
directional. That is, whether there is relatively unrestricted diffusion in one
particular direction.

Mathematically, FA is defined as the normalized variance of the eigen-values of
the tensor:

.. math::

        FA = \sqrt{\frac{1}{2}\frac{(\lambda_1-\lambda_2)^2+(\lambda_1-
                    \lambda_3)^2+(\lambda_2-\lambda_3)^2}{\lambda_1^2+
                    \lambda_2^2+\lambda_3^2}}

Note that FA should be interperted carefully. It may be an indication of the
density of packing of fibers in a voxel, and the amount of myelin wrapping these
axons, but it is not always a measure of "tissue integrity". For example, FA
may decrease in locations in which there is fanning of white matter fibers, or
where more than one population of white matter fibers crosses.

"""

from dipy.reconst.dti import fractional_anisotropy

FA = fractional_anisotropy(tenfit.evals)

"""
In the background of the image the fitting will not be accurate there is no
signal and possibly we will find FA values with nans (not a number). We can
easily remove these in the following way.
"""

FA[np.isnan(FA)] = 0

"""
Saving the FA images is very easy using nibabel. We need the FA volume and the
affine matrix which transform the image's coordinates to the world coordinates.
"""

fa_img = nib.Nifti1Image(FA, img.get_affine())
nib.save(fa_img, 'tensor_fa.nii.gz')

"""
You can now see the result with any nifti viewer or check it slice by slice
using matplotlib_'s imshow. In the same way you can save the eigen values the
eigen vectors or any other properties of the Tensor.
"""

evecs_img = nib.Nifti1Image(tenfit.evecs, img.get_affine())
nib.save(evecs_img, 'tensor_evecs.nii.gz')

"""
Finally lets try to visualize the orientation distribution functions of a small
rectangular area around the middle of our datasets.
"""

i,j,k,w = np.array(data2.shape) / 2
data_small  = data2[i-5:i+5, j-5:j+5, k-2:k+2]
from dipy.data import get_sphere
sphere = get_sphere('symmetric724')

from dipy.viz import fvtk
r = fvtk.ren()
fvtk.add(r, fvtk.sphere_funcs(tenmodel.fit(data_small).odf(sphere),
							  sphere, colormap=None))

print('Saving illustration as tensor_odfs.png')
fvtk.record(r, n_frames=1, out_path='tensor_odfs.png', size=(600, 600))

"""
.. figure:: tensor_odfs.png
   :align: center

   **Tensor ODFs**.

Note that while the tensor model is an accurate and reliable model of the
diffusion signal in the white matter, it has the drawback that it only has one
principal diffusion direction. Therefore, in locations in the brain that
contain multiple fiber populations crossing each other, the tensor model may
indicate that the principal diffusion direction is intermediate to these
directions. Therefore, using the principal diffusion direction for tracking in
these locations may be misleading and may lead to errors in defining the
tracks. Fortunately, other reconstruction methods can be used to represent the
diffusion and fiber orientations in those locations. These are presented in
other examples.

.. [Basser1994] Basser PJ, Mattielo J, LeBihan (1994). MR diffusion tensor
                spectroscopy and imaging.

.. include:: ../links_names.inc

"""
<|endoftext|>


<commit_before>import re
import os
from SimPEG import Mesh, Utils
import numpy as np
from . import BaseMag
from . import Magnetics

class MagneticsDriver_Inv(object):
    """docstring for MagneticsDriver_Inv"""

    def __init__(self, input_file=None):
        if input_file is not None:
            self.basePath = os.path.sep.join(input_file.split(
                                             os.path.sep)[:-1])
            if len(self.basePath) > 0:
                self.basePath += os.path.sep
            self.readDriverFile(input_file.split(os.path.sep)[-1])

    def readDriverFile(self, input_file):
        """
            Read input files for forward modeling MAG data with integral form
            INPUT:
            input_file: File name containing the forward parameter

            OUTPUT:
            mshfile
            obsfile
            topofile
            start model
            ref model
            mag model
            weightfile
            chi_target
            as, ax ,ay, az
            upper, lower bounds
            lp, lqx, lqy, lqz

            # All files should be in the working directory,
            # otherwise the path must be specified.

        """

        fid = open(self.basePath + input_file, 'r')

        # Line 1: Mesh
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        mshfile = l_input[1].rstrip()

        # Line 2: Observation file
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        obsfile = l_input[1].rstrip()

        # Line 3: Topo, active-dyn, active-static
        topofile = None
        staticInput = None

        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'TOPO':
            topofile = l_input[1].rstrip()

        elif l_input[0] == 'VALUE':
            staticInput = float(l_input[1])

        elif l_input[0] == 'FILE':
            staticInput = l_input[1].rstrip()

        # Line 4: Starting model
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'VALUE':
            mstart = float(l_input[1])

        elif l_input[0] == 'FILE':
            mstart = l_input[1].rstrip()

        # Line 5: Reference model
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'VALUE':
            mref = float(l_input[1])

        elif l_input[0] == 'FILE':
            mref = l_input[1].rstrip()

        # Line 6: Magnetization model
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'DEFAULT':
            magfile = None

        elif l_input[0] == 'FILE':
            magfile = l_input[1].rstrip()

        # Line 7: Cell weights
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'DEFAULT':
            wgtfile = []

        elif l_input[0] == 'FILE':
            wgtfile = l_input[1].rstrip()

        # Line 8: Target chi-factor
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'DEFAULT':
            chi = 1.

        elif l_input[0] == 'VALUE':
            chi = float(l_input[1])

        # Line 9: Alpha values
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'VALUE':

            val = np.array(l_input[1:5])
            alphas = val.astype(np.float)

        elif l_input[0] == 'DEFAULT':

            alphas = np.ones(4)

        # Line 10: Bounds
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'VALUE':
            val = np.array(l_input[1:3])
            bounds = val.astype(np.float)

        elif l_input[0] == 'FILE':
            bounds = l_input[1].rstrip()

        else:
            bounds = [-np.inf, np.inf]

        # Line 11: Norms
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'VALUE':
            val = np.array(l_input[1:6])
            lpnorms = val.astype(np.float)

        elif l_input[0] == 'FILE':
            lpnorms = l_input[1].rstrip()

        # Line 12: Treshold values
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'VALUE':
            val = np.array(l_input[1:3])
            eps = val.astype(np.float)

        elif l_input[0] == 'DEFAULT':
            eps = None

        self.mshfile = mshfile
        self.obsfile = obsfile
        self.topofile = topofile
        self.mstart = mstart
        self._mrefInput = mref
        self._staticInput = staticInput
        self.magfile = magfile
        self.wgtfile = wgtfile
        self.chi = chi
        self.alphas = alphas
        self.bounds = bounds
        self.lpnorms = lpnorms
        self.eps = eps

    @property
    def mesh(self):
        if getattr(self, '_mesh', None) is None:
            self._mesh = Mesh.TensorMesh.readUBC(self.basePath + self.mshfile)
        return self._mesh

    @property
    def survey(self):
        if getattr(self, '_survey', None) is None:
            self._survey = self.readMagneticsObservations(self.obsfile)
        return self._survey

    @property
    def activeCells(self):
        if getattr(self, '_activeCells', None) is None:
            if getattr(self, 'topofile', None) is not None:
                topo = np.genfromtxt(self.basePath + self.topofile,
                                     skip_header=1)
                # Find the active cells
                active = Utils.surface2ind_topo(self.mesh, topo, 'N')

            elif isinstance(self._staticInput, float):
                active = self.m0 != self._staticInput

            else:
                # Read from file active cells with 0:air, 1:dynamic, -1 static
                active = self.activeModel != 0

            inds = np.asarray([inds for inds,
                               elem in enumerate(active, 1)
                               if elem], dtype=int) - 1

            self._activeCells = inds

            # Reduce m0 to active space
            if len(self.m0) > len(self._activeCells):
                self._m0 = self.m0[self._activeCells]

        return self._activeCells

    @property
    def staticCells(self):
        if getattr(self, '_staticCells', None) is None:

            # Cells with value 1 in active model are dynamic
            staticCells = self.activeModel[self._activeCells] == -1

            inds = np.asarray([inds for inds,
                               elem in enumerate(staticCells, 1)
                               if elem], dtype=int) - 1

            self._staticCells = inds

        return self._staticCells

    @property
    def dynamicCells(self):
        if getattr(self, '_dynamicCells', None) is None:

            # Cells with value 1 in active model are dynamic
            dynamicCells = self.activeModel[self._activeCells] == 1

            inds = np.asarray([inds for inds,
                               elem in enumerate(dynamicCells, 1)
                               if elem], dtype=int) - 1

            self._dynamicCells = inds

        return self._dynamicCells

    @property
    def nC(self):
        if getattr(self, '_nC', None) is None:
            self._nC = len(self.activeCells)
        return self._nC

    @property
    def m0(self):
        if getattr(self, '_m0', None) is None:
            if isinstance(self.mstart, float):
                self._m0 = np.ones(self.nC) * self.mstart
            else:
                self._m0 = Mesh.TensorMesh.readModelUBC(self.mesh,
                                                        self.basePath +
                                                        self.mstart)

        return self._m0

    @property
    def mref(self):
        if getattr(self, '_mref', None) is None:
            if isinstance(self._mrefInput, float):
                self._mref = np.ones(self.nC) * self._mrefInput
            else:
                self._mref = Mesh.TensorMesh.readModelUBC(self.mesh,
                                                          self.basePath +
                                                          self._mrefInput)

                # Reduce to active space
                self._mref = self._mref[self._activeCells]

        return self._mref

    @property
    def activeModel(self):
        if getattr(self, '_activeModel', None) is None:
            if self._staticInput == 'FILE':
                # Read from file active cells with 0:air, 1:dynamic, -1 static
                self._activeModel = Mesh.TensorMesh.readModelUBC(self.mesh, self.basePath + self._staticInput)

            else:
                self._activeModel = np.ones(self._mesh.nC)

        return self._activeModel

    @property
    def magnetizationModel(self):
        """
            magnetization vector
        """

        if getattr(self, 'magfile', None) is None:

            M = Magnetics.dipazm_2_xyz(np.ones(self.nC) *
                                       self.survey.srcField.param[1],
                                       np.ones(self.nC) *
                                       self.survey.srcField.param[2])

        else:

            with open(self.basePath + self.magfile) as f:
                magmodel = f.read()

            magmodel = magmodel.splitlines()
            M = []

            for line in magmodel:
                M.append(map(float, line.split()))

            # Convert list to 2d array
            M = np.vstack(M)

            # Cycle through three components and permute from UBC to SimPEG
            for ii in range(3):
                m = np.reshape(M[:, ii],
                               (self.mesh.nCz, self.mesh.nCx, self.mesh.nCy),
                               order='F')

                m = m[::-1, :, :]
                m = np.transpose(m, (1, 2, 0))
                M[:, ii] = Utils.mkvc(m)

        self._M = M

        return self._M

    def readMagneticsObservations(self, obs_file):
        """
            Read and write UBC mag file format

            INPUT:
            :param fileName, path to the UBC obs mag file

            OUTPUT:
            :param survey
            :param M, magnetization orentiaton (MI, MD)
        """

        fid = open(self.basePath + obs_file, 'r')

        # First line has the inclination,declination and amplitude of B0
        line = fid.readline()
        B = np.array(line.split(), dtype=float)

        # Second line has the magnetization orientation and a flag
        line = fid.readline()
        M = np.array(line.split(), dtype=float)

        # Third line has the number of rows
        line = fid.readline()
        ndat = np.array(line.split(), dtype=int)

        # Pre-allocate space for obsx, obsy, obsz, data, uncert
        line = fid.readline()
        temp = np.array(line.split(), dtype=float)

        d = np.zeros(ndat, dtype=float)
        wd = np.zeros(ndat, dtype=float)
        locXYZ = np.zeros((ndat[0], 3), dtype=float)

        for ii in range(ndat):

            temp = np.array(line.split(), dtype=float)
            locXYZ[ii, :] = temp[:3]

            if len(temp) > 3:
                d[ii] = temp[3]

                if len(temp) == 5:
                    wd[ii] = temp[4]

            line = fid.readline()

        rxLoc = BaseMag.RxObs(locXYZ)
        srcField = BaseMag.SrcField([rxLoc], param=(B[2], B[0], B[1]))
        survey = BaseMag.LinearSurvey(srcField)
        survey.dobs = d
        survey.std = wd
        return survey
<commit_msg>ensure we are using ints in the MagDriver<commit_after>import re
import os
from SimPEG import Mesh, Utils
import numpy as np
from . import BaseMag
from . import Magnetics

class MagneticsDriver_Inv(object):
    """docstring for MagneticsDriver_Inv"""

    def __init__(self, input_file=None):
        if input_file is not None:
            self.basePath = os.path.sep.join(input_file.split(
                                             os.path.sep)[:-1])
            if len(self.basePath) > 0:
                self.basePath += os.path.sep
            self.readDriverFile(input_file.split(os.path.sep)[-1])

    def readDriverFile(self, input_file):
        """
            Read input files for forward modeling MAG data with integral form
            INPUT:
            input_file: File name containing the forward parameter

            OUTPUT:
            mshfile
            obsfile
            topofile
            start model
            ref model
            mag model
            weightfile
            chi_target
            as, ax ,ay, az
            upper, lower bounds
            lp, lqx, lqy, lqz

            # All files should be in the working directory,
            # otherwise the path must be specified.

        """

        fid = open(self.basePath + input_file, 'r')

        # Line 1: Mesh
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        mshfile = l_input[1].rstrip()

        # Line 2: Observation file
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        obsfile = l_input[1].rstrip()

        # Line 3: Topo, active-dyn, active-static
        topofile = None
        staticInput = None

        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'TOPO':
            topofile = l_input[1].rstrip()

        elif l_input[0] == 'VALUE':
            staticInput = float(l_input[1])

        elif l_input[0] == 'FILE':
            staticInput = l_input[1].rstrip()

        # Line 4: Starting model
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'VALUE':
            mstart = float(l_input[1])

        elif l_input[0] == 'FILE':
            mstart = l_input[1].rstrip()

        # Line 5: Reference model
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'VALUE':
            mref = float(l_input[1])

        elif l_input[0] == 'FILE':
            mref = l_input[1].rstrip()

        # Line 6: Magnetization model
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'DEFAULT':
            magfile = None

        elif l_input[0] == 'FILE':
            magfile = l_input[1].rstrip()

        # Line 7: Cell weights
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'DEFAULT':
            wgtfile = []

        elif l_input[0] == 'FILE':
            wgtfile = l_input[1].rstrip()

        # Line 8: Target chi-factor
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'DEFAULT':
            chi = 1.

        elif l_input[0] == 'VALUE':
            chi = float(l_input[1])

        # Line 9: Alpha values
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'VALUE':

            val = np.array(l_input[1:5])
            alphas = val.astype(np.float)

        elif l_input[0] == 'DEFAULT':

            alphas = np.ones(4)

        # Line 10: Bounds
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'VALUE':
            val = np.array(l_input[1:3])
            bounds = val.astype(np.float)

        elif l_input[0] == 'FILE':
            bounds = l_input[1].rstrip()

        else:
            bounds = [-np.inf, np.inf]

        # Line 11: Norms
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'VALUE':
            val = np.array(l_input[1:6])
            lpnorms = val.astype(np.float)

        elif l_input[0] == 'FILE':
            lpnorms = l_input[1].rstrip()

        # Line 12: Treshold values
        line = fid.readline()
        l_input = re.split('[!\s]', line)
        if l_input[0] == 'VALUE':
            val = np.array(l_input[1:3])
            eps = val.astype(np.float)

        elif l_input[0] == 'DEFAULT':
            eps = None

        self.mshfile = mshfile
        self.obsfile = obsfile
        self.topofile = topofile
        self.mstart = mstart
        self._mrefInput = mref
        self._staticInput = staticInput
        self.magfile = magfile
        self.wgtfile = wgtfile
        self.chi = chi
        self.alphas = alphas
        self.bounds = bounds
        self.lpnorms = lpnorms
        self.eps = eps

    @property
    def mesh(self):
        if getattr(self, '_mesh', None) is None:
            self._mesh = Mesh.TensorMesh.readUBC(self.basePath + self.mshfile)
        return self._mesh

    @property
    def survey(self):
        if getattr(self, '_survey', None) is None:
            self._survey = self.readMagneticsObservations(self.obsfile)
        return self._survey

    @property
    def activeCells(self):
        if getattr(self, '_activeCells', None) is None:
            if getattr(self, 'topofile', None) is not None:
                topo = np.genfromtxt(self.basePath + self.topofile,
                                     skip_header=1)
                # Find the active cells
                active = Utils.surface2ind_topo(self.mesh, topo, 'N')

            elif isinstance(self._staticInput, float):
                active = self.m0 != self._staticInput

            else:
                # Read from file active cells with 0:air, 1:dynamic, -1 static
                active = self.activeModel != 0

            inds = np.asarray([inds for inds,
                               elem in enumerate(active, 1)
                               if elem], dtype=int) - 1

            self._activeCells = inds

            # Reduce m0 to active space
            if len(self.m0) > len(self._activeCells):
                self._m0 = self.m0[self._activeCells]

        return self._activeCells

    @property
    def staticCells(self):
        if getattr(self, '_staticCells', None) is None:

            # Cells with value 1 in active model are dynamic
            staticCells = self.activeModel[self._activeCells] == -1

            inds = np.asarray([inds for inds,
                               elem in enumerate(staticCells, 1)
                               if elem], dtype=int) - 1

            self._staticCells = inds

        return self._staticCells

    @property
    def dynamicCells(self):
        if getattr(self, '_dynamicCells', None) is None:

            # Cells with value 1 in active model are dynamic
            dynamicCells = self.activeModel[self._activeCells] == 1

            inds = np.asarray([inds for inds,
                               elem in enumerate(dynamicCells, 1)
                               if elem], dtype=int) - 1

            self._dynamicCells = inds

        return self._dynamicCells

    @property
    def nC(self):
        if getattr(self, '_nC', None) is None:
            self._nC = len(self.activeCells)
        return self._nC

    @property
    def m0(self):
        if getattr(self, '_m0', None) is None:
            if isinstance(self.mstart, float):
                self._m0 = np.ones(self.nC) * self.mstart
            else:
                self._m0 = Mesh.TensorMesh.readModelUBC(self.mesh,
                                                        self.basePath +
                                                        self.mstart)

        return self._m0

    @property
    def mref(self):
        if getattr(self, '_mref', None) is None:
            if isinstance(self._mrefInput, float):
                self._mref = np.ones(self.nC) * self._mrefInput
            else:
                self._mref = Mesh.TensorMesh.readModelUBC(self.mesh,
                                                          self.basePath +
                                                          self._mrefInput)

                # Reduce to active space
                self._mref = self._mref[self._activeCells]

        return self._mref

    @property
    def activeModel(self):
        if getattr(self, '_activeModel', None) is None:
            if self._staticInput == 'FILE':
                # Read from file active cells with 0:air, 1:dynamic, -1 static
                self._activeModel = Mesh.TensorMesh.readModelUBC(self.mesh, self.basePath + self._staticInput)

            else:
                self._activeModel = np.ones(self._mesh.nC)

        return self._activeModel

    @property
    def magnetizationModel(self):
        """
            magnetization vector
        """

        if getattr(self, 'magfile', None) is None:

            M = Magnetics.dipazm_2_xyz(np.ones(self.nC) *
                                       self.survey.srcField.param[1],
                                       np.ones(self.nC) *
                                       self.survey.srcField.param[2])

        else:

            with open(self.basePath + self.magfile) as f:
                magmodel = f.read()

            magmodel = magmodel.splitlines()
            M = []

            for line in magmodel:
                M.append(map(float, line.split()))

            # Convert list to 2d array
            M = np.vstack(M)

            # Cycle through three components and permute from UBC to SimPEG
            for ii in range(3):
                m = np.reshape(M[:, ii],
                               (self.mesh.nCz, self.mesh.nCx, self.mesh.nCy),
                               order='F')

                m = m[::-1, :, :]
                m = np.transpose(m, (1, 2, 0))
                M[:, ii] = Utils.mkvc(m)

        self._M = M

        return self._M

    def readMagneticsObservations(self, obs_file):
        """
            Read and write UBC mag file format

            INPUT:
            :param fileName, path to the UBC obs mag file

            OUTPUT:
            :param survey
            :param M, magnetization orentiaton (MI, MD)
        """

        fid = open(self.basePath + obs_file, 'r')

        # First line has the inclination,declination and amplitude of B0
        line = fid.readline()
        B = np.array(line.split(), dtype=float)

        # Second line has the magnetization orientation and a flag
        line = fid.readline()
        M = np.array(line.split(), dtype=float)

        # Third line has the number of rows
        line = fid.readline()
        ndat = int(line.strip())

        # Pre-allocate space for obsx, obsy, obsz, data, uncert
        line = fid.readline()
        temp = np.array(line.split(), dtype=float)

        d = np.zeros(ndat, dtype=float)
        wd = np.zeros(ndat, dtype=float)
        locXYZ = np.zeros((ndat, 3), dtype=float)

        for ii in range(ndat):

            temp = np.array(line.split(), dtype=float)
            locXYZ[ii, :] = temp[:3]

            if len(temp) > 3:
                d[ii] = temp[3]

                if len(temp) == 5:
                    wd[ii] = temp[4]

            line = fid.readline()

        rxLoc = BaseMag.RxObs(locXYZ)
        srcField = BaseMag.SrcField([rxLoc], param=(B[2], B[0], B[1]))
        survey = BaseMag.LinearSurvey(srcField)
        survey.dobs = d
        survey.std = wd
        return survey
<|endoftext|>


<commit_before>"""@package genrandforsorting
This module simply generates random numbers to use within the sorting
algorithms.
"""

import random
import sys
import time

class genrandforsorting:
    def __init__(self,
                 seed=1,
                 minimum=0,
                 maximum=10000,
                 count=100,
                 filename='out.txt'):
        """Initializes genrandforsorting class."""
        self.seed = seed
        self.min = minimum
        self.max = maximum
        self.count = count
        self.filename = filename

    def genrandom(self):
        """Generates random data in output file."""
        random.seed(self.seed)
        with open(self.filename, 'w') as f:
            for i in range(self.count):
                f.write(str(random.randint(self.min, self.max)))
                f.write('\n')

def main():
    """Main function, executed if this file is run."""
    genrandforsorting(time.time()).genrandom()
    return 0

if __name__ == "__main__":
    """Execute only if run as a script."""
    main()
<commit_msg>Renamed default file for input.<commit_after>"""@package genrandforsorting
This module simply generates random numbers to use within the sorting
algorithms.
"""

import random
import sys
import time

class genrandforsorting:
    def __init__(self,
                 seed=1,
                 minimum=0,
                 maximum=10000,
                 count=100,
                 filename='in.txt'):
        """Initializes genrandforsorting class."""
        self.seed = seed
        self.min = minimum
        self.max = maximum
        self.count = count
        self.filename = filename

    def genrandom(self):
        """Generates random data in output file."""
        random.seed(self.seed)
        with open(self.filename, 'w') as f:
            for i in range(self.count):
                f.write(str(random.randint(self.min, self.max)))
                f.write('\n')

def main():
    """Main function, executed if this file is run."""
    genrandforsorting(time.time()).genrandom()
    return 0

if __name__ == "__main__":
    """Execute only if run as a script."""
    main()
<|endoftext|>


<commit_before>import datetime

date = raw_input('Creating new file, enter ISO date (if today then just press Enter): ')
if len(date) == 0:
	print '\nDate set to: '
	date = datetime.date.today()
	print date
title = raw_input('\nEnter title: ')

print '\nFile is generated!'<commit_msg>automating blog post file creation<commit_after>#Just a simple script to automate the YML frontmatter in new posts
import datetime

date = raw_input('Creating new file, enter ISO date (if today then just press Enter): ')
if len(date) == 0:
	print '\nDate set to: '
	date = datetime.date.today()
	date = date.strftime('%Y-%m-%d')
	print date

title = raw_input('\nEnter title: ')

fileName= date + '-' + title.replace(" ", "_").lower() + '.md'
print fileName
file = open(fileName, "w")

file.write('---\nlayout: post\ntitle: ' + title + '\n---\n\n\n\n')

print '\nFile is generated!'<|endoftext|>


<commit_before># Software License Agreement (BSD License)
#
# Copyright (c) 2012, Willow Garage, Inc.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
#  * Redistributions of source code must retain the above copyright
#    notice, this list of conditions and the following disclaimer.
#  * Redistributions in binary form must reproduce the above
#    copyright notice, this list of conditions and the following
#    disclaimer in the documentation and/or other materials provided
#    with the distribution.
#  * Neither the name of Willow Garage, Inc. nor the names of its
#    contributors may be used to endorse or promote products derived
#    from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.

from threading import Lock
from functools import partial
from rospy import loginfo
from rosbridge_library.capability import Capability
from rosbridge_library.internal.subscribers import manager
from rosbridge_library.internal.subscription_modifiers import MessageHandler
from rosbridge_library.internal.pngcompression import encode
try:
    from ujson import dumps
except ImportError:
    try:
        from simplejson import dumps
    except ImportError:
        from json import dumps

class Subscription():
    """ Keeps track of the clients multiple calls to subscribe.

    Chooses the most appropriate settings to send messages """

    def __init__(self, client_id, topic, publish):
        """ Create a subscription for the specified client on the specified
        topic, with callback publish

        Keyword arguments:
        client_id -- the ID of the client making this subscription
        topic     -- the name of the topic to subscribe to
        publish   -- the callback function for incoming messages

        """
        self.client_id = client_id
        self.topic = topic
        self.publish = publish

        self.clients = {}

        self.handler = MessageHandler(None, self._publish)
        self.handler_lock = Lock()
        self.update_params()

    def unregister(self):
        """ Unsubscribes this subscription and cleans up resources """
        manager.unsubscribe(self.client_id, self.topic)
        with self.handler_lock:
            self.handler.finish()
        self.clients.clear()

    def subscribe(self, sid=None, msg_type=None, throttle_rate=0,
                  queue_length=0, fragment_size=None, compression="none"):
        """ Add another client's subscription request

        If there are multiple calls to subscribe, the values actually used for
        queue_length, fragment_size, compression and throttle_rate are
        chosen to encompass all subscriptions' requirements

        Keyword arguments:
        sid             -- the subscription id from the client
        msg_type        -- the type of the message to subscribe to
        throttle_rate   -- the minimum time (in ms) allowed between messages
        being sent.  If multiple subscriptions, the lower of these is used
        queue_length    -- the number of messages that can be buffered.  If
        multiple subscriptions, the lower of these is used
        fragment_size   -- None if no fragmentation, or the maximum length of
        allowed outgoing messages
        compression     -- "none" if no compression, or some other value if
        compression is to be used (current valid values are 'png')

         """
        # Subscribe with the manager. This will propagate any exceptions
        manager.subscribe(self.client_id, self.topic, self.on_msg, msg_type)

        client_details = {
            "throttle_rate": throttle_rate,
            "queue_length": queue_length,
            "fragment_size": fragment_size,
            "compression": compression
        }

        self.clients[sid] = client_details

        self.update_params()

    def unsubscribe(self, sid=None):
        """ Unsubscribe this particular client's subscription

        Keyword arguments:
        sid -- the individual subscription id.  If None, all are unsubscribed

        """
        if sid is None:
            self.clients.clear()
        elif sid in self.clients:
            del self.clients[sid]

        if not self.is_empty():
            self.update_params()

    def is_empty(self):
        """ Return true if there are no subscriptions currently """
        return len(self.clients) == 0

    def _publish(self, message):
        """ Internal method to propagate published messages to the registered
        publish callback """
        self.publish(message, self.fragment_size, self.compression)

    def on_msg(self, msg):
        """ Raw callback called by subscription manager for all incoming
        messages.

        Incoming messages are passed to the message handler which may drop,
        buffer, or propagate the message

        """
        with self.handler_lock:
            self.handler.handle_message(msg)

    def update_params(self):
        """ Determine the 'lowest common denominator' params to satisfy all
        subscribed clients. """
        if len(self.clients) == 0:
            self.throttle_rate = 0
            self.queue_length = 0
            self.fragment_size = None
            self.compression = "none"
            return

        def f(fieldname):
            return [x[fieldname] for x in self.clients.values()]

        self.throttle_rate = min(f("throttle_rate"))
        self.queue_length = min(f("queue_length"))
        frags = [x for x in f("fragment_size") if x != None]
        if frags == []:
            self.fragment_size = None
        else:
            self.fragment_size = min(frags)
        self.compression = "png" if "png" in f("compression") else "none"

        with self.handler_lock:
            self.handler = self.handler.set_throttle_rate(self.throttle_rate)
            self.handler = self.handler.set_queue_length(self.queue_length)

class Subscribe(Capability):

    subscribe_msg_fields = [(True, "topic", (str, unicode)), (False, "type", (str, unicode)),
        (False, "throttle_rate", int), (False, "fragment_size", int),
        (False, "queue_length", int), (False, "compression", (str, unicode))]
    unsubscribe_msg_fields = [(True, "topic", (str, unicode))]

    def __init__(self, protocol):
        # Call superclass constructor
        Capability.__init__(self, protocol)

        # Register the operations that this capability provides
        protocol.register_operation("subscribe", self.subscribe)
        protocol.register_operation("unsubscribe", self.unsubscribe)

        self._subscriptions = {}

    def subscribe(self, msg):
        # Pull out the ID
        sid = msg.get("id", None)

        # Check the args
        self.basic_type_check(msg, self.subscribe_msg_fields)

        # Make the subscription
        topic = msg["topic"]
        if not topic in self._subscriptions:
            client_id = self.protocol.client_id
            cb = partial(self.publish, topic)
            self._subscriptions[topic] = Subscription(client_id, topic, cb)

        # Register the subscriber
        subscribe_args = {
          "sid": sid,
          "msg_type": msg.get("type", None),
          "throttle_rate": msg.get("throttle_rate", 0),
          "fragment_size": msg.get("fragment_size", None),
          "queue_length": msg.get("queue_length", 0),
          "compression": msg.get("compression", "none")
        }
        self._subscriptions[topic].subscribe(**subscribe_args)

        self.protocol.log("info", "Subscribed to %s" % topic)

    def unsubscribe(self, msg):
        # Pull out the ID
        sid = msg.get("id", None)

        self.basic_type_check(msg, self.unsubscribe_msg_fields)

        topic = msg["topic"]
        if topic not in self._subscriptions:
            return
        self._subscriptions[topic].unsubscribe(sid)

        if self._subscriptions[topic].is_empty():
            self._subscriptions[topic].unregister()
            del self._subscriptions[topic]

        self.protocol.log("info", "Unsubscribed from %s" % topic)

    def publish(self, topic, message, fragment_size=None, compression="none"):
        """ Publish a message to the client

        Keyword arguments:
        topic   -- the topic to publish the message on
        message -- a dict of key-value pairs. Will be wrapped in a message with
        opcode publish
        fragment_size -- (optional) fragment the serialized message into msgs
        with payloads not greater than this value
        compression   -- (optional) compress the message. valid values are
        'png' and 'none'

        """
        # TODO: fragmentation, proper ids
        outgoing_msg = {"op": "publish", "topic": topic, "msg": message}
        if compression=="png":
            outgoing_msg_dumped = dumps(outgoing_msg)
            outgoing_msg = {"op": "png", "data": encode(outgoing_msg_dumped)}
        self.protocol.send(outgoing_msg)

    def finish(self):
        for subscription in self._subscriptions.values():
            subscription.unregister()
        self._subscriptions.clear()
        self.protocol.unregister_operation("subscribe")
        self.protocol.unregister_operation("unsubscribe")
<commit_msg>avoiding racing condition<commit_after># Software License Agreement (BSD License)
#
# Copyright (c) 2012, Willow Garage, Inc.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
#  * Redistributions of source code must retain the above copyright
#    notice, this list of conditions and the following disclaimer.
#  * Redistributions in binary form must reproduce the above
#    copyright notice, this list of conditions and the following
#    disclaimer in the documentation and/or other materials provided
#    with the distribution.
#  * Neither the name of Willow Garage, Inc. nor the names of its
#    contributors may be used to endorse or promote products derived
#    from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.

from threading import Lock
from functools import partial
from rospy import loginfo
from rosbridge_library.capability import Capability
from rosbridge_library.internal.subscribers import manager
from rosbridge_library.internal.subscription_modifiers import MessageHandler
from rosbridge_library.internal.pngcompression import encode
try:
    from ujson import dumps
except ImportError:
    try:
        from simplejson import dumps
    except ImportError:
        from json import dumps

class Subscription():
    """ Keeps track of the clients multiple calls to subscribe.

    Chooses the most appropriate settings to send messages """

    def __init__(self, client_id, topic, publish):
        """ Create a subscription for the specified client on the specified
        topic, with callback publish

        Keyword arguments:
        client_id -- the ID of the client making this subscription
        topic     -- the name of the topic to subscribe to
        publish   -- the callback function for incoming messages

        """
        self.client_id = client_id
        self.topic = topic
        self.publish = publish

        self.clients = {}

        self.handler = MessageHandler(None, self._publish)
        self.handler_lock = Lock()
        self.update_params()

    def unregister(self):
        """ Unsubscribes this subscription and cleans up resources """
        manager.unsubscribe(self.client_id, self.topic)
        with self.handler_lock:
            self.handler.finish()
        self.clients.clear()

    def subscribe(self, sid=None, msg_type=None, throttle_rate=0,
                  queue_length=0, fragment_size=None, compression="none"):
        """ Add another client's subscription request

        If there are multiple calls to subscribe, the values actually used for
        queue_length, fragment_size, compression and throttle_rate are
        chosen to encompass all subscriptions' requirements

        Keyword arguments:
        sid             -- the subscription id from the client
        msg_type        -- the type of the message to subscribe to
        throttle_rate   -- the minimum time (in ms) allowed between messages
        being sent.  If multiple subscriptions, the lower of these is used
        queue_length    -- the number of messages that can be buffered.  If
        multiple subscriptions, the lower of these is used
        fragment_size   -- None if no fragmentation, or the maximum length of
        allowed outgoing messages
        compression     -- "none" if no compression, or some other value if
        compression is to be used (current valid values are 'png')

         """

        client_details = {
            "throttle_rate": throttle_rate,
            "queue_length": queue_length,
            "fragment_size": fragment_size,
            "compression": compression
        }

        self.clients[sid] = client_details

        self.update_params()

        # Subscribe with the manager. This will propagate any exceptions
        manager.subscribe(self.client_id, self.topic, self.on_msg, msg_type)

    def unsubscribe(self, sid=None):
        """ Unsubscribe this particular client's subscription

        Keyword arguments:
        sid -- the individual subscription id.  If None, all are unsubscribed

        """
        if sid is None:
            self.clients.clear()
        elif sid in self.clients:
            del self.clients[sid]

        if not self.is_empty():
            self.update_params()

    def is_empty(self):
        """ Return true if there are no subscriptions currently """
        return len(self.clients) == 0

    def _publish(self, message):
        """ Internal method to propagate published messages to the registered
        publish callback """
        self.publish(message, self.fragment_size, self.compression)

    def on_msg(self, msg):
        """ Raw callback called by subscription manager for all incoming
        messages.

        Incoming messages are passed to the message handler which may drop,
        buffer, or propagate the message

        """
        with self.handler_lock:
            self.handler.handle_message(msg)

    def update_params(self):
        """ Determine the 'lowest common denominator' params to satisfy all
        subscribed clients. """
        if len(self.clients) == 0:
            self.throttle_rate = 0
            self.queue_length = 0
            self.fragment_size = None
            self.compression = "none"
            return

        def f(fieldname):
            return [x[fieldname] for x in self.clients.values()]

        self.throttle_rate = min(f("throttle_rate"))
        self.queue_length = min(f("queue_length"))
        frags = [x for x in f("fragment_size") if x != None]
        if frags == []:
            self.fragment_size = None
        else:
            self.fragment_size = min(frags)
        self.compression = "png" if "png" in f("compression") else "none"

        with self.handler_lock:
            self.handler = self.handler.set_throttle_rate(self.throttle_rate)
            self.handler = self.handler.set_queue_length(self.queue_length)

class Subscribe(Capability):

    subscribe_msg_fields = [(True, "topic", (str, unicode)), (False, "type", (str, unicode)),
        (False, "throttle_rate", int), (False, "fragment_size", int),
        (False, "queue_length", int), (False, "compression", (str, unicode))]
    unsubscribe_msg_fields = [(True, "topic", (str, unicode))]

    def __init__(self, protocol):
        # Call superclass constructor
        Capability.__init__(self, protocol)

        # Register the operations that this capability provides
        protocol.register_operation("subscribe", self.subscribe)
        protocol.register_operation("unsubscribe", self.unsubscribe)

        self._subscriptions = {}

    def subscribe(self, msg):
        # Pull out the ID
        sid = msg.get("id", None)

        # Check the args
        self.basic_type_check(msg, self.subscribe_msg_fields)

        # Make the subscription
        topic = msg["topic"]
        if not topic in self._subscriptions:
            client_id = self.protocol.client_id
            cb = partial(self.publish, topic)
            self._subscriptions[topic] = Subscription(client_id, topic, cb)

        # Register the subscriber
        subscribe_args = {
          "sid": sid,
          "msg_type": msg.get("type", None),
          "throttle_rate": msg.get("throttle_rate", 0),
          "fragment_size": msg.get("fragment_size", None),
          "queue_length": msg.get("queue_length", 0),
          "compression": msg.get("compression", "none")
        }
        self._subscriptions[topic].subscribe(**subscribe_args)

        self.protocol.log("info", "Subscribed to %s" % topic)

    def unsubscribe(self, msg):
        # Pull out the ID
        sid = msg.get("id", None)

        self.basic_type_check(msg, self.unsubscribe_msg_fields)

        topic = msg["topic"]
        if topic not in self._subscriptions:
            return
        self._subscriptions[topic].unsubscribe(sid)

        if self._subscriptions[topic].is_empty():
            self._subscriptions[topic].unregister()
            del self._subscriptions[topic]

        self.protocol.log("info", "Unsubscribed from %s" % topic)

    def publish(self, topic, message, fragment_size=None, compression="none"):
        """ Publish a message to the client

        Keyword arguments:
        topic   -- the topic to publish the message on
        message -- a dict of key-value pairs. Will be wrapped in a message with
        opcode publish
        fragment_size -- (optional) fragment the serialized message into msgs
        with payloads not greater than this value
        compression   -- (optional) compress the message. valid values are
        'png' and 'none'

        """
        # TODO: fragmentation, proper ids
        outgoing_msg = {"op": "publish", "topic": topic, "msg": message}
        if compression=="png":
            outgoing_msg_dumped = dumps(outgoing_msg)
            outgoing_msg = {"op": "png", "data": encode(outgoing_msg_dumped)}
        self.protocol.send(outgoing_msg)

    def finish(self):
        for subscription in self._subscriptions.values():
            subscription.unregister()
        self._subscriptions.clear()
        self.protocol.unregister_operation("subscribe")
        self.protocol.unregister_operation("unsubscribe")
<|endoftext|>


<commit_before># -*- coding: utf-8 -*-
"""Classes for handling posts and archives."""

import os
import re

import datetime as dtmod
try:
    from urlparse import urljoin
except ImportError:
    from urllib.parse import urljoin
from datetime import datetime
from unicodedata import normalize

from docutils import nodes
from docutils.io import StringOutput

from sphinx.util.osutil import relative_uri
from sphinx.environment import dummy_reporter

def slugify(string):
    """Slugify *s*."""

    try:
        string = unicode(string)
        string = normalize('NFKD', string).encode('ascii', 'ignore')
    except NameError:
        string = str(string)
        string = normalize('NFKD', string).encode('ascii', 'ignore').decode()

    string = re.sub(r'[^\w\s-]', '', string).strip().lower()
    return re.sub(r'[-\s]+', '-', string)

DEBUG = True
CONFIG = [
    # name, default, rebuild
    ('blog_path', 'blog', True),
    ('blog_title', 'Blog', True),
    ('blog_baseurl', None, True),

    ('blog_feed_archives', False, True),
    ('blog_feed_fulltext', False, True),
    ('blog_feed_subtitle', None, True),

    ('blog_authors', {}, True),
    ('blog_default_author', None, True),
    ('blog_locations', {}, True),
    ('blog_default_location', None, True),
    ('blog_languages', {}, True),
    ('blog_default_language', None, True),

    ('fontawesome_link_cdn', False, True),
    ('fontawesome_included', False, True),
    ('fontawesome_css_file', False, True),

    ('post_date_format', '%b %d, %Y', True),
    ('post_date_format_short', '%b %d', True),
    ('post_auto_image', 0, True),
    ('post_auto_excerpt', 1, True),
    ('post_redirect_refresh', 5, True),
    ('post_always_section', False, True),

    ('disqus_shortname', None, True),
    ('disqus_drafts', False, True),
    ('disqus_pages', False, True),

    ('skip_pickling', False, True),
]

TOMORROW = datetime.today() + dtmod.timedelta(1)
FUTURE = datetime(9999, 12, 31)

class Blog(object):

    """Handle blog operations."""

    # using a shared state
    _dict = {}

    def __init__(self, app=None):

        self.__dict__ = self._dict
        if not self._dict:
            self._init(app)

    def _init(self, app):
        """Instantiate Blog."""

        self.app = app
        self.config = {}

        # std domain of for creating references to posts and archives
        self.std_domain = domain = self.app.env.domains['std']

        # get configuration from Sphinx app
        for opt in CONFIG:
            self.config[opt[0]] = getattr(app.config, opt[0])

        opt = self.config['blog_default_author']
        if opt is not None and not isinstance(opt, list):
            self.config['blog_default_author'] = [opt]

        opt = self.config['blog_default_location']
        if opt is not None and not isinstance(opt, list):
            self.config['blog_default_location'] = [opt]

        opt = self.config['blog_default_language']
        if opt is not None and not isinstance(opt, list):
            self.config['blog_default_language'] = [opt]

        # blog catalog contains all posts
        self.blog = Catalog(self, 'blog', 'blog', None)

        # contains post collections by year
        self.archive = Catalog(self, 'archive', 'archive', None)
        domain.data['labels']['blog-archives'] = (
            self.archive.docname, '', 'Archives')

        self.catalogs = cat = {}  # catalogs of user set labels
        self.tags = cat['tags'] = Catalog(self, 'tags', 'tag', 'tag')
        domain.data['labels']['blog-tags'] = (
            self.tags.docname, '', 'Tags')

        self.author = cat['author'] = Catalog(self, 'author',
            'author', 'author')
        domain.data['labels']['blog-authors'] = (
            self.author.docname, '', 'Authors')

        self.location = cat['location'] = Catalog(self, 'location',
            'location',  'location')
        domain.data['labels']['blog-locations'] = (
            self.location.docname, '', 'Locations')

        self.language = cat['language'] = Catalog(self, 'language',
            'language',  'language')
        domain.data['labels']['blog-languages'] = (
            self.language.docname, '', 'Languages')

        self.category = cat['category'] = Catalog(self, 'category',
            'category', 'category')
        domain.data['labels']['blog-categories'] = (
            self.category.docname, '', 'Categories')

        for catname in ['author', 'location', 'language']:
            catalog = self.catalogs[catname]
            items = self.config['blog_' + catname + 's'].items()
            for label, (name, link) in items:
                catalog[label] = Collection(catalog, label, name, link)

        self.posts = self.blog['post'] = Collection(self.blog, 'post',
            'Posts', path=self.blog_path)
        self.drafts = self.blog['draft'] = Collection(self.blog, 'draft',
            'Drafts', path=os.path.join(self.blog_path, 'drafts'))

        # add references to posts and drafts
        # e.g. :ref:`blog-posts`
        domain.data['labels']['blog-posts'] = (
            os.path.join(self.config['blog_path'], 'index'), '', 'Posts')
        domain.data['labels']['blog-drafts'] = (
            os.path.join(self.config['blog_path'], 'drafts', 'index'), '',
                         'Drafts')
        domain.data['labels']['blog-feed'] = (
            os.path.join(self.config['blog_path'], 'atom.xml'), '',
            self.blog_title + ' Feed')

        # set some internal configuration options
        self.config['cloud_size'] = 5
        self.config['fontawesome'] = (self.config['fontawesome_included'] or
                                      self.config['fontawesome_link_cdn'] or
                                      self.config['fontawesome_css_file'])

    def __getattr__(self, name):

        try:
            attr = self.config[name]
        except KeyError:
            raise AttributeError('ABlog has no configuration option {}'
                                 .format(repr(name)))
        return attr

    def __getitem__(self, key):

        return self.posts[key] or self.drafts[key]

    def __contains__(self, item):

        return item in self.posts or item in self.drafts

    def __len__(self):

        return len(self.posts)

    def __nonzero__(self):

        return len(self) > 0

    @property
    def feed_path(self):
        """RSS feed page name."""

        return os.path.join(self.blog_path, 'rss.xml')

    def register(self, docname, info):
        """Register post *docname*."""

        post = Post(self, docname, info)
        if post.date and post.date < TOMORROW:
            self.posts.add(post)
        else:
            self.drafts.add(post)
        for catalog in self.catalogs.values():
            catalog.add(post)

    def recent(self, num, docname=None, **labels):
        """Yield *num* recent posts, excluding the one with `docname`."""

        for i, post in enumerate(self.posts):
            if post.docname == docname:
                num += 1
                continue
            if i == num:
                return
            yield post

    def link_posts(self):
        """Link posts after sorting them post by published date."""

        posts = [post for post in self.posts if post.order == 1]
        posts.sort()
        posts[0].prev = posts[-1].next = None
        for i in range(1, len(posts)):
            post = posts[i]
            posts[i - 1].next = post
            post.prev = posts[i - 1]

    def page_id(self, pagename):
        """Page identifier for Disqus."""

        if self.config['blog_baseurl']:
            if pagename.endswith('index'):
                pagename = pagename[:-5]
            pagename = pagename.strip('/')
            return '/' + pagename + ('/' if pagename else '')

    def page_url(self, pagename):
        """Page url for Disqus."""

        if self.config['blog_baseurl']:
            url = urljoin(self.config['blog_baseurl'], pagename)
            if url.endswith('index'):
                url = url[:-5]
            return url

def html_builder_write_doc(self, docname, doctree):
    """Part of :meth:`sphinx.builders.html.StandaloneHTMLBuilder.write_doc`
    method used to convert *doctree* to HTML."""

    destination = StringOutput(encoding='utf-8')
    doctree.settings = self.docsettings

    self.secnumbers = {}
    self.imgpath = relative_uri(self.get_target_uri(docname), '_images')
    self.dlpath = relative_uri(self.get_target_uri(docname), '_downloads')
    self.current_docname = docname
    self.docwriter.write(doctree, destination)
    self.docwriter.assemble_parts()
    return self.docwriter.parts['fragment']

class Post(object):

    """Handle post metadata."""

    def __init__(self, ablog, docname, info):

        self.ablog = ablog
        self.docname = docname
        self.section = info['section']
        self.order = info['order']
        self.date = date = info['date']
        self.update = info['update']
        self.published = date and date < TOMORROW
        self.draft = not self.published
        self.title = info['title']
        self.excerpt = info['excerpt']
        self.doctree = info['doctree']
        self._next = self._prev = -1

        #self.language = info.get('language')

        # archives
        self.blog = []
        if self.published:
            self.tags = info.get('tags')
            self.author = info.get('author')
            self.category = info.get('category')
            self.location = info.get('location')
            self.language = info.get('language')

            if not self.author and ablog.blog_default_author:
                self.author = ablog.blog_default_author
            if not self.location and ablog.blog_default_location:
                self.location = ablog.blog_default_location
            if not self.language and ablog.blog_default_language:
                self.language = ablog.blog_default_language

            self.archive = self.ablog.archive[self.date.year]
            self.archive.add(self)

        else:
            self.tags = info.get('tags')
            self.author = info.get('author')
            self.category = info.get('category')
            self.location = info.get('location')
            self.language = info.get('language')
            self.archive = []

        self.redirect = info.get('redirect')

        self.options = info

    def __lt__(self, other):

        return ((self.date or FUTURE), self.title) < (
                (other.date or FUTURE), other.title)

    def to_html(self, pagename, fulltext=False):
        """Return excerpt as HTML after resolving references with respect to
        *pagename*."""

        if fulltext:
            doctree = nodes.document({}, dummy_reporter)
            deepcopy = self.doctree.deepcopy()
            if isinstance(deepcopy, nodes.document):
                doctree.extend(deepcopy.children)
            else:
                doctree.append(deepcopy)
        else:
            doctree = nodes.document({}, dummy_reporter)
            for node in self.excerpt:
                doctree.append(node.deepcopy())
        app = self.ablog.app
        app.env.resolve_references(doctree, pagename, app.builder)
        html = html_builder_write_doc(app.builder, pagename, doctree)
        return html

    @property
    def next(self):
        """Next published post in chronological order."""

        if self._next == -1:
            self.ablog.link_posts()
        return self._next

    @next.setter
    def next(self, post):
        """Set next published post in chronological order."""

        self._next = post

    @property
    def prev(self):
        """Previous published post in chronological order."""

        if self._prev == -1:
            self.ablog.link_posts()
        return self._prev

    @prev.setter
    def prev(self, post):
        """Set previous published post in chronological order."""

        self._prev = post

class Catalog(object):

    """Handles collections of posts."""

    def __init__(self, blog, name, xref, path):

        self.blog = blog
        self.name = name
        self.xref = xref # for creating labels, e.g. `tag-python`
        self.collections = {}

        if path:
            self.path = self.docname = os.path.join(blog.blog_path, path)
        else:
            self.path = self.docname = blog.blog_path

        self._lens = None
        self._minmax = None

    def __getitem__(self, name):

        try:
            return self.collections[name]
        except KeyError:
            return self.collections.setdefault(name, Collection(self, name))

    def __setitem__(self, name, item):

        self.collections[name] = item

    def __len__(self):

        return sum(len(coll) for coll in self)

    def __nonzero__(self):

        return len(self) > 0

    def __iter__(self):

        keys = list(self.collections)
        keys.sort()
        for key in keys:
            yield self.collections[key]

    def add(self, post):
        """Add post to appropriate collection(s) and replace collections
        labels with collection objects."""

        colls = []
        for label in getattr(post, self.name, []):
            coll = self[label]
            if post.published:
                coll.add(post)
            colls.append(coll)
        setattr(post, self.name, colls)

    def minmax(self):
        """Return minimum and maximum sizes of collections."""

        if self._lens is None or len(self._lens) != len(self.collections):
            self._lens = [len(coll) for coll in self.collections.values()
                          if len(coll)]
            self._minmax = min(self._lens), max(self._lens)
        return self._minmax

class Collection(object):

    """Posts sharing a label, i.e. tag, category, author, or location."""

    def __init__(self, catalog, label, name=None, href=None, path=None):

        self.catalog = catalog
        self.label = label
        self.name = name or self.label
        self.href = href

        self._posts = {}

        self._path = path
        self.xref = self.catalog.xref + '-' + slugify(label)
        self._slug = None
        self._html = None

        self.catalog.blog.std_domain.data['labels'][self.xref] = (
            self.docname, self.xref, self.name)

    def __str__(self):

        return str(self.name)

    def __len__(self):

        return len(self._posts)

    def __nonzero__(self):

        return len(self) > 0

    def __unicode__(self):

        return unicode(self.name)

    def __iter__(self):

        posts = list(self._posts.values())
        posts.sort(reverse=True)
        for post in posts:
            yield post

    def __getitem__(self, key):

        return self._posts.get(key)

    def __contains__(self, item):

        return item in self._posts

    def add(self, post):
        """Add post to the collection."""

        post_name = post.docname
        if post.section:
            post_name += '#' + post.section
        self._posts[post_name] = post

    @property
    def relsize(self):
        """Relative size used in tag clouds. Relat"""

        if len(self.catalog) == 1:
            return 3
        min_, max_ = self.catalog.minmax()
        size = int((len(self) - min_) / max(max_ - min_, 1) *
                   (self.catalog.blog.cloud_size - 1)) + 1
        return size

    @property
    def docname(self):
        """Collection page document name."""

        if self._path is None:
            self._path = os.path.join(self.catalog.path, slugify(self.name))
        return self._path

    path = docname
<commit_msg>Dropping h1 from post html txt in atom feeds (fixed issue #21).)<commit_after># -*- coding: utf-8 -*-
"""Classes for handling posts and archives."""

import os
import re

import datetime as dtmod
try:
    from urlparse import urljoin
except ImportError:
    from urllib.parse import urljoin
from datetime import datetime
from unicodedata import normalize

from docutils import nodes
from docutils.io import StringOutput

from sphinx.util.osutil import relative_uri
from sphinx.environment import dummy_reporter

def slugify(string):
    """Slugify *s*."""

    try:
        string = unicode(string)
        string = normalize('NFKD', string).encode('ascii', 'ignore')
    except NameError:
        string = str(string)
        string = normalize('NFKD', string).encode('ascii', 'ignore').decode()

    string = re.sub(r'[^\w\s-]', '', string).strip().lower()
    return re.sub(r'[-\s]+', '-', string)

DEBUG = True
CONFIG = [
    # name, default, rebuild
    ('blog_path', 'blog', True),
    ('blog_title', 'Blog', True),
    ('blog_baseurl', None, True),

    ('blog_feed_archives', False, True),
    ('blog_feed_fulltext', False, True),
    ('blog_feed_subtitle', None, True),

    ('blog_authors', {}, True),
    ('blog_default_author', None, True),
    ('blog_locations', {}, True),
    ('blog_default_location', None, True),
    ('blog_languages', {}, True),
    ('blog_default_language', None, True),

    ('fontawesome_link_cdn', False, True),
    ('fontawesome_included', False, True),
    ('fontawesome_css_file', False, True),

    ('post_date_format', '%b %d, %Y', True),
    ('post_date_format_short', '%b %d', True),
    ('post_auto_image', 0, True),
    ('post_auto_excerpt', 1, True),
    ('post_redirect_refresh', 5, True),
    ('post_always_section', False, True),

    ('disqus_shortname', None, True),
    ('disqus_drafts', False, True),
    ('disqus_pages', False, True),

    ('skip_pickling', False, True),
]

TOMORROW = datetime.today() + dtmod.timedelta(1)
FUTURE = datetime(9999, 12, 31)

class Blog(object):

    """Handle blog operations."""

    # using a shared state
    _dict = {}

    def __init__(self, app=None):

        self.__dict__ = self._dict
        if not self._dict:
            self._init(app)

    def _init(self, app):
        """Instantiate Blog."""

        self.app = app
        self.config = {}

        # std domain of for creating references to posts and archives
        self.std_domain = domain = self.app.env.domains['std']

        # get configuration from Sphinx app
        for opt in CONFIG:
            self.config[opt[0]] = getattr(app.config, opt[0])

        opt = self.config['blog_default_author']
        if opt is not None and not isinstance(opt, list):
            self.config['blog_default_author'] = [opt]

        opt = self.config['blog_default_location']
        if opt is not None and not isinstance(opt, list):
            self.config['blog_default_location'] = [opt]

        opt = self.config['blog_default_language']
        if opt is not None and not isinstance(opt, list):
            self.config['blog_default_language'] = [opt]

        # blog catalog contains all posts
        self.blog = Catalog(self, 'blog', 'blog', None)

        # contains post collections by year
        self.archive = Catalog(self, 'archive', 'archive', None)
        domain.data['labels']['blog-archives'] = (
            self.archive.docname, '', 'Archives')

        self.catalogs = cat = {}  # catalogs of user set labels
        self.tags = cat['tags'] = Catalog(self, 'tags', 'tag', 'tag')
        domain.data['labels']['blog-tags'] = (
            self.tags.docname, '', 'Tags')

        self.author = cat['author'] = Catalog(self, 'author',
            'author', 'author')
        domain.data['labels']['blog-authors'] = (
            self.author.docname, '', 'Authors')

        self.location = cat['location'] = Catalog(self, 'location',
            'location',  'location')
        domain.data['labels']['blog-locations'] = (
            self.location.docname, '', 'Locations')

        self.language = cat['language'] = Catalog(self, 'language',
            'language',  'language')
        domain.data['labels']['blog-languages'] = (
            self.language.docname, '', 'Languages')

        self.category = cat['category'] = Catalog(self, 'category',
            'category', 'category')
        domain.data['labels']['blog-categories'] = (
            self.category.docname, '', 'Categories')

        for catname in ['author', 'location', 'language']:
            catalog = self.catalogs[catname]
            items = self.config['blog_' + catname + 's'].items()
            for label, (name, link) in items:
                catalog[label] = Collection(catalog, label, name, link)

        self.posts = self.blog['post'] = Collection(self.blog, 'post',
            'Posts', path=self.blog_path)
        self.drafts = self.blog['draft'] = Collection(self.blog, 'draft',
            'Drafts', path=os.path.join(self.blog_path, 'drafts'))

        # add references to posts and drafts
        # e.g. :ref:`blog-posts`
        domain.data['labels']['blog-posts'] = (
            os.path.join(self.config['blog_path'], 'index'), '', 'Posts')
        domain.data['labels']['blog-drafts'] = (
            os.path.join(self.config['blog_path'], 'drafts', 'index'), '',
                         'Drafts')
        domain.data['labels']['blog-feed'] = (
            os.path.join(self.config['blog_path'], 'atom.xml'), '',
            self.blog_title + ' Feed')

        # set some internal configuration options
        self.config['cloud_size'] = 5
        self.config['fontawesome'] = (self.config['fontawesome_included'] or
                                      self.config['fontawesome_link_cdn'] or
                                      self.config['fontawesome_css_file'])

    def __getattr__(self, name):

        try:
            attr = self.config[name]
        except KeyError:
            raise AttributeError('ABlog has no configuration option {}'
                                 .format(repr(name)))
        return attr

    def __getitem__(self, key):

        return self.posts[key] or self.drafts[key]

    def __contains__(self, item):

        return item in self.posts or item in self.drafts

    def __len__(self):

        return len(self.posts)

    def __nonzero__(self):

        return len(self) > 0

    @property
    def feed_path(self):
        """RSS feed page name."""

        return os.path.join(self.blog_path, 'rss.xml')

    def register(self, docname, info):
        """Register post *docname*."""

        post = Post(self, docname, info)
        if post.date and post.date < TOMORROW:
            self.posts.add(post)
        else:
            self.drafts.add(post)
        for catalog in self.catalogs.values():
            catalog.add(post)

    def recent(self, num, docname=None, **labels):
        """Yield *num* recent posts, excluding the one with `docname`."""

        for i, post in enumerate(self.posts):
            if post.docname == docname:
                num += 1
                continue
            if i == num:
                return
            yield post

    def link_posts(self):
        """Link posts after sorting them post by published date."""

        posts = [post for post in self.posts if post.order == 1]
        posts.sort()
        posts[0].prev = posts[-1].next = None
        for i in range(1, len(posts)):
            post = posts[i]
            posts[i - 1].next = post
            post.prev = posts[i - 1]

    def page_id(self, pagename):
        """Page identifier for Disqus."""

        if self.config['blog_baseurl']:
            if pagename.endswith('index'):
                pagename = pagename[:-5]
            pagename = pagename.strip('/')
            return '/' + pagename + ('/' if pagename else '')

    def page_url(self, pagename):
        """Page url for Disqus."""

        if self.config['blog_baseurl']:
            url = urljoin(self.config['blog_baseurl'], pagename)
            if url.endswith('index'):
                url = url[:-5]
            return url

def html_builder_write_doc(self, docname, doctree):
    """Part of :meth:`sphinx.builders.html.StandaloneHTMLBuilder.write_doc`
    method used to convert *doctree* to HTML."""

    destination = StringOutput(encoding='utf-8')
    doctree.settings = self.docsettings

    self.secnumbers = {}
    self.imgpath = relative_uri(self.get_target_uri(docname), '_images')
    self.dlpath = relative_uri(self.get_target_uri(docname), '_downloads')
    self.current_docname = docname
    self.docwriter.write(doctree, destination)
    self.docwriter.assemble_parts()
    return self.docwriter.parts['fragment']

class Post(object):

    """Handle post metadata."""

    def __init__(self, ablog, docname, info):

        self.ablog = ablog
        self.docname = docname
        self.section = info['section']
        self.order = info['order']
        self.date = date = info['date']
        self.update = info['update']
        self.published = date and date < TOMORROW
        self.draft = not self.published
        self.title = info['title']
        self.excerpt = info['excerpt']
        self.doctree = info['doctree']
        self._next = self._prev = -1

        #self.language = info.get('language')

        # archives
        self.blog = []
        if self.published:
            self.tags = info.get('tags')
            self.author = info.get('author')
            self.category = info.get('category')
            self.location = info.get('location')
            self.language = info.get('language')

            if not self.author and ablog.blog_default_author:
                self.author = ablog.blog_default_author
            if not self.location and ablog.blog_default_location:
                self.location = ablog.blog_default_location
            if not self.language and ablog.blog_default_language:
                self.language = ablog.blog_default_language

            self.archive = self.ablog.archive[self.date.year]
            self.archive.add(self)

        else:
            self.tags = info.get('tags')
            self.author = info.get('author')
            self.category = info.get('category')
            self.location = info.get('location')
            self.language = info.get('language')
            self.archive = []

        self.redirect = info.get('redirect')

        self.options = info

    def __lt__(self, other):

        return ((self.date or FUTURE), self.title) < (
                (other.date or FUTURE), other.title)

    def to_html(self, pagename, fulltext=False, drop_h1=True):
        """Return excerpt as HTML after resolving references with respect to
        *pagename*."""

        if fulltext:
            doctree = nodes.document({}, dummy_reporter)
            deepcopy = self.doctree.deepcopy()
            if isinstance(deepcopy, nodes.document):
                doctree.extend(deepcopy.children)
            else:
                doctree.append(deepcopy)
        else:
            doctree = nodes.document({}, dummy_reporter)
            for node in self.excerpt:
                doctree.append(node.deepcopy())
        app = self.ablog.app
        app.env.resolve_references(doctree, pagename, app.builder)
        html = html_builder_write_doc(app.builder, pagename, doctree)

        if drop_h1:
            html = re.sub('<h1>(.*?)</h1>', '', html, count=int(drop_h1))
        return html

    @property
    def next(self):
        """Next published post in chronological order."""

        if self._next == -1:
            self.ablog.link_posts()
        return self._next

    @next.setter
    def next(self, post):
        """Set next published post in chronological order."""

        self._next = post

    @property
    def prev(self):
        """Previous published post in chronological order."""

        if self._prev == -1:
            self.ablog.link_posts()
        return self._prev

    @prev.setter
    def prev(self, post):
        """Set previous published post in chronological order."""

        self._prev = post

class Catalog(object):

    """Handles collections of posts."""

    def __init__(self, blog, name, xref, path):

        self.blog = blog
        self.name = name
        self.xref = xref # for creating labels, e.g. `tag-python`
        self.collections = {}

        if path:
            self.path = self.docname = os.path.join(blog.blog_path, path)
        else:
            self.path = self.docname = blog.blog_path

        self._lens = None
        self._minmax = None

    def __getitem__(self, name):

        try:
            return self.collections[name]
        except KeyError:
            return self.collections.setdefault(name, Collection(self, name))

    def __setitem__(self, name, item):

        self.collections[name] = item

    def __len__(self):

        return sum(len(coll) for coll in self)

    def __nonzero__(self):

        return len(self) > 0

    def __iter__(self):

        keys = list(self.collections)
        keys.sort()
        for key in keys:
            yield self.collections[key]

    def add(self, post):
        """Add post to appropriate collection(s) and replace collections
        labels with collection objects."""

        colls = []
        for label in getattr(post, self.name, []):
            coll = self[label]
            if post.published:
                coll.add(post)
            colls.append(coll)
        setattr(post, self.name, colls)

    def minmax(self):
        """Return minimum and maximum sizes of collections."""

        if self._lens is None or len(self._lens) != len(self.collections):
            self._lens = [len(coll) for coll in self.collections.values()
                          if len(coll)]
            self._minmax = min(self._lens), max(self._lens)
        return self._minmax

class Collection(object):

    """Posts sharing a label, i.e. tag, category, author, or location."""

    def __init__(self, catalog, label, name=None, href=None, path=None):

        self.catalog = catalog
        self.label = label
        self.name = name or self.label
        self.href = href

        self._posts = {}

        self._path = path
        self.xref = self.catalog.xref + '-' + slugify(label)
        self._slug = None
        self._html = None

        self.catalog.blog.std_domain.data['labels'][self.xref] = (
            self.docname, self.xref, self.name)

    def __str__(self):

        return str(self.name)

    def __len__(self):

        return len(self._posts)

    def __nonzero__(self):

        return len(self) > 0

    def __unicode__(self):

        return unicode(self.name)

    def __iter__(self):

        posts = list(self._posts.values())
        posts.sort(reverse=True)
        for post in posts:
            yield post

    def __getitem__(self, key):

        return self._posts.get(key)

    def __contains__(self, item):

        return item in self._posts

    def add(self, post):
        """Add post to the collection."""

        post_name = post.docname
        if post.section:
            post_name += '#' + post.section
        self._posts[post_name] = post

    @property
    def relsize(self):
        """Relative size used in tag clouds. Relat"""

        if len(self.catalog) == 1:
            return 3
        min_, max_ = self.catalog.minmax()
        size = int((len(self) - min_) / max(max_ - min_, 1) *
                   (self.catalog.blog.cloud_size - 1)) + 1
        return size

    @property
    def docname(self):
        """Collection page document name."""

        if self._path is None:
            self._path = os.path.join(self.catalog.path, slugify(self.name))
        return self._path

    path = docname
<|endoftext|>