<commit_before>""" Joblib is a set of tools to provide **lightweight pipelining in
Python**. In particular, joblib offers:

  1. transparent disk-caching of the output values and lazy re-evaluation
     (memoize pattern)

  2. easy simple parallel computing

  3. logging and tracing of the execution

Joblib is optimized to be **fast** and **robust** in particular on large
data and has specific optimizations for `numpy` arrays. It is
**BSD-licensed**.

    ============================== ==========================================
    **User documentation**:        http://packages.python.org/joblib

    **Download packages**:         http://pypi.python.org/pypi/joblib

    **Source code**:               http://github.com/joblib/joblib

    **Report issues**:             http://github.com/joblib/joblib/issues
    ============================== ==========================================

Vision
--------

The vision is to provide tools to easily achieve better performance and
reproducibility when working with long running jobs. In addition, Joblib
can also be used to provide a light-weight make replacement or caching
solution.

 *  **Avoid computing twice the same thing**: code is rerun over an
    over, for instance when prototyping computational-heavy jobs (as in
    scientific development), but hand-crafted solution to aleviate this
    issue is error-prone and often leads to unreproducible results

 *  **Persist to disk transparently**: persisting in an efficient way
    arbitrary objects containing large data is hard. In addition,
    hand-written persistence does not link easily the file on disk to the
    execution context of the original Python object. As a result, it is
    challenging to resume a application status or computational job, eg
    after a crash.

It strives to address these problems while **leaving your code and your
flow control as unmodified as possible** (no framework, no new
paradigms).

Main features
------------------

1) **Transparent and fast disk-caching of output value:** a memoize or
   make-like functionality for Python functions that works well for
   arbitrary Python objects, including very large numpy arrays. Separate
   persistence and flow-execution logic from domain logic or algorithmic
   code by writing the operations as a set of steps with well-defined
   inputs and  outputs: Python functions. Joblib can save their
   computation to disk and rerun it only if necessary::

      >>> from joblib import Memory
      >>> mem = Memory(cachedir='/tmp/joblib')
      >>> import numpy as np
      >>> a = np.vander(np.arange(3))
      >>> square = mem.cache(np.square)
      >>> b = square(a)
      ________________________________________________________________________________
      [Memory] Calling square...
      square(array([[0, 0, 1],
             [1, 1, 1],
             [4, 2, 1]]))
      ___________________________________________________________square - 0.0s, 0.0min

      >>> c = square(a)
      >>> # The above call did not trigger an evaluation

2) **Embarrassingly parallel helper:** to make is easy to write readable
   parallel code and debug it quickly:

      >>> from joblib import Parallel, delayed
      >>> from math import sqrt
      >>> Parallel(n_jobs=1)(delayed(sqrt)(i**2) for i in range(10))
      [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0]

3) **Logging/tracing:** The different functionalities will
   progressively acquire better logging mechanism to help track what
   has been ran, and capture I/O easily. In addition, Joblib will
   provide a few I/O primitives, to easily define define logging and
   display streams, and provide a way of compiling a report.
   We want to be able to quickly inspect what has been run.

..
    >>> import shutil ; shutil.rmtree('/tmp/joblib/')

"""

__version__ = '0.4.4'

from .memory import Memory
from .logger import PrintTime, Logger
from .hashing import hash
from .numpy_pickle import dump, load
from .parallel import Parallel, delayed

<commit_msg>MISC: Better link for download<commit_after>""" Joblib is a set of tools to provide **lightweight pipelining in
Python**. In particular, joblib offers:

  1. transparent disk-caching of the output values and lazy re-evaluation
     (memoize pattern)

  2. easy simple parallel computing

  3. logging and tracing of the execution

Joblib is optimized to be **fast** and **robust** in particular on large
data and has specific optimizations for `numpy` arrays. It is
**BSD-licensed**.

    ============================== ==============================================
    **User documentation**:        http://packages.python.org/joblib

    **Download packages**:         http://pypi.python.org/pypi/joblib#downloads

    **Source code**:               http://github.com/joblib/joblib

    **Report issues**:             http://github.com/joblib/joblib/issues
    ============================== ==============================================

Vision
--------

The vision is to provide tools to easily achieve better performance and
reproducibility when working with long running jobs. In addition, Joblib
can also be used to provide a light-weight make replacement or caching
solution.

 *  **Avoid computing twice the same thing**: code is rerun over an
    over, for instance when prototyping computational-heavy jobs (as in
    scientific development), but hand-crafted solution to aleviate this
    issue is error-prone and often leads to unreproducible results

 *  **Persist to disk transparently**: persisting in an efficient way
    arbitrary objects containing large data is hard. In addition,
    hand-written persistence does not link easily the file on disk to the
    execution context of the original Python object. As a result, it is
    challenging to resume a application status or computational job, eg
    after a crash.

It strives to address these problems while **leaving your code and your
flow control as unmodified as possible** (no framework, no new
paradigms).

Main features
------------------

1) **Transparent and fast disk-caching of output value:** a memoize or
   make-like functionality for Python functions that works well for
   arbitrary Python objects, including very large numpy arrays. Separate
   persistence and flow-execution logic from domain logic or algorithmic
   code by writing the operations as a set of steps with well-defined
   inputs and  outputs: Python functions. Joblib can save their
   computation to disk and rerun it only if necessary::

      >>> from joblib import Memory
      >>> mem = Memory(cachedir='/tmp/joblib')
      >>> import numpy as np
      >>> a = np.vander(np.arange(3))
      >>> square = mem.cache(np.square)
      >>> b = square(a)
      ________________________________________________________________________________
      [Memory] Calling square...
      square(array([[0, 0, 1],
             [1, 1, 1],
             [4, 2, 1]]))
      ___________________________________________________________square - 0.0s, 0.0min

      >>> c = square(a)
      >>> # The above call did not trigger an evaluation

2) **Embarrassingly parallel helper:** to make is easy to write readable
   parallel code and debug it quickly:

      >>> from joblib import Parallel, delayed
      >>> from math import sqrt
      >>> Parallel(n_jobs=1)(delayed(sqrt)(i**2) for i in range(10))
      [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0]

3) **Logging/tracing:** The different functionalities will
   progressively acquire better logging mechanism to help track what
   has been ran, and capture I/O easily. In addition, Joblib will
   provide a few I/O primitives, to easily define define logging and
   display streams, and provide a way of compiling a report.
   We want to be able to quickly inspect what has been run.

..
    >>> import shutil ; shutil.rmtree('/tmp/joblib/')

"""

__version__ = '0.4.4'

from .memory import Memory
from .logger import PrintTime, Logger
from .hashing import hash
from .numpy_pickle import dump, load
from .parallel import Parallel, delayed

<|endoftext|>


<commit_before>__author__ = 'Chick Markley'

import sys
import ast
import math
import time

def singleton(cls):
    instance = cls()
    instance.__call__ = lambda: instance
    return instance

class Timer(object):
    """
    Context manager for timing sections of code.
    """

    class _stopwatch(object):
        """
        Times a particular key.
        """

        def __init__(self):
            self.x = 0.0
            self.x2 = 0.0
            self.nLaps = 0

        def start(self):
            self._start = time.time()
            self.nLaps += 1

        def lap(self):
            t = time.time() - self._start
            self.x += t
            self.x2 += t * t

        def mean(self):
            if self.nLaps == 0:
                return 0
            else:
                return self.x / float(self.nLaps)

        def stddev(self):
            if self.nLaps == 0:
                return 0
            else:
                avg = self.mean()
                return math.sqrt(self.x2 / float(self.nLaps) - (avg * avg))

        def dict(self):
            return {
                "nTrials": self.nLaps,
                "mean": self.mean(),
                "stddev": self.stddev(),
            }

    def __init__(self):
        self.watches = {}

    def __enter__(self):
        self.watches[self._key].start()

    def __exit__(self, sym_type, value, traceback):
        self.watches[self._key].lap()

    def __call__(self, key):
        if key not in self.watches:
            self.watches[key] = Timer._stopwatch()
        self._key = key
        return self

    def dict(self):
        return {k: self.watches[k].dict() for k in self.watches}

class DotManager(object):
    """
    take ast and return an ipython image file
    """

    @staticmethod
    def dot_ast_to_image(ast_node):
        import io
        from IPython.display import Image

        from  ctree.dotgen import to_dot

        dot_text = to_dot(ast_node)

        return DotManager.dot_text_to_image(dot_text)

    @staticmethod
    def dot_text_to_image(text):
        from IPython.display import Image

        dot_output = DotManager.run_dot(text)
        return Image(dot_output, embed=True)

    @staticmethod
    def run_dot(code, options=None, output_format='png'):
        # mostly copied from sphinx.ext.graphviz.render_dot
        import os
        from subprocess import Popen, PIPE
        from sphinx.util.osutil import EPIPE, EINVAL

        if not options:
            options = []
        dot_args = ['dot'] + options + ['-T', output_format]
        if os.name == 'nt':
            # Avoid opening shell window.
            # * https://github.com/tkf/ipython-hierarchymagic/issues/1
            # * http://stackoverflow.com/a/2935727/727827
            p = Popen(dot_args, stdout=PIPE, stdin=PIPE, stderr=PIPE,
                      creationflags=0x08000000)
        else:
            p = Popen(dot_args, stdout=PIPE, stdin=PIPE, stderr=PIPE)
        went_wrong = False
        try:
            # Graphviz may close standard input when an error occurs,
            # resulting in a broken pipe on communicate()
            stdout, stderr = p.communicate(code.encode('utf-8'))
        except (OSError, IOError) as err:
            if err.errno != EPIPE:
                raise
            went_wrong = True
        except IOError as err:
            if err.errno != EINVAL:
                raise
            went_wrong = True
        if went_wrong:
            # in this case, read the standard output and standard error streams
            # directly, to get the error message(s)
            stdout, stderr = p.stdout.read(), p.stderr.read()
            p.wait()
        if p.returncode != 0:
            raise RuntimeError('dot exited with error:\n[stderr]\n{0}'
                               .format(stderr.decode('utf-8')))
        return stdout
<commit_msg>remove Timer and DotManager classes<commit_after>__author__ = 'Chick Markley'

def singleton(cls):
    instance = cls()
    instance.__call__ = lambda: instance
    return instance
<|endoftext|>


<commit_before>"""CVS interface for CVSGit."""

import os.path
import re
import time

from subprocess import Popen, PIPE

from cvsgit.changeset import ChangeSetGenerator
from cvsgit.rcs import RCSFile
from cvsgit.i18n import _
from cvsgit.term import NoProgress

def split_cvs_source(dirname):
    """Split <dirname> into CVSROOT and module paths.
    """
    cvsroot = dirname
    module = ''
    while True:
        parent = os.path.dirname(cvsroot)
        if cvsroot == parent:
            raise TypeError, _('not a CVS repository path (%s): %s') \
                % (_('no CVSROOT within nor above'), dirname)
        if os.path.isdir(os.path.join(cvsroot, 'CVSROOT')):
            return (cvsroot, module,)
        if module == '':
            module = os.path.basename(cvsroot)
        else:
            module = os.path.join(os.path.basename(cvsroot), module)
        cvsroot = parent

class CVS(object):
    """Represents a CVS repository.
    """

    def __init__(self, dirname, metadb):
        self.metadb = metadb

        # 'dirname' is a local filesystem path pointing at the root of
        # a CVS repository or at a module within.  If it is a module
        # path, operations will be limited to that module.  Otherwise,
        # the module path will be empty and operations will by default
        # apply to the whole repository.
        #
        # In any case, the repository path is always determined by
        # recursively following the parents of 'dirname' until a a
        # sub-directory named 'CVSROOT' is found.  Repository path and
        # module path are available for reading in the instance
        # attributes 'root' and 'module', respectively.

        if not os.path.isdir(dirname):
            raise TypeError, _('not a CVS repository path (%s): %s') \
                % (_('not even a directory'), dirname)

        # Convert to absolute pathname, so that our logic doesn't
        # break if anyone uses os.chdir() and so that the path
        # traversal below can always assume an absolute path.
        dirname = os.path.abspath(dirname)

        # Split 'dirname' into self.root and self.module and also
        # set self.prefix to the full absolute module path.
        self.root, self.module = split_cvs_source(dirname)
        if self.module == '':
            self.prefix = self.root
        else:
            self.prefix = os.path.join(self.root, self.module)

        self.localid = None
        self.parse_options()

        self.statcache = {}
        self._rcs_keyword_re = re.compile('\$([^$:\r\n]+)[^$\r\n]*\$')
        self._rcs_headerfix_re = re.compile(' ([^ ]+,v) ')
        self._rcs_strip_attic_re = re.compile('(Attic/)?([^/]+),v$')

    def parse_options(self):
        """Extract relevant information from the CVSROOT/options file,
        if it exsits.  At the moment, the only information that is
        used from the file is the custom Id tag keyword 'tag'."""

        filename = os.path.join(self.root, 'CVSROOT', 'options')
        if not os.path.isfile(filename):
            return

        f = file(filename, 'r')
        try:
            for line in f.readlines():
                option, value = line.split('=', 2)
                if option == 'tag':
                    self.localid = value.strip()
        finally:
            f.close()

    # Helper function to check with the statcache and the stat()
    # system call if a file or directory is unmodified.
    def _unmodified(self, path):
        st = os.stat(os.path.join(self.prefix, path))
        identity = (st.st_mtime, st.st_size,)
        return self.statcache.has_key(path) and \
                self.statcache[path] == identity

    def changed_rcs_filenames(self, progress=None):
        """Return the list of RCS filenames which need to be scanned for
        new changes to import.
        """
        if not progress:
            progress = NoProgress()

        with progress:
            return self._changed_rcs_filenames(progress=progress)

    def _changed_rcs_filenames(self, progress=None):
        # Helper function to raise the OSError reported by os.walk().
        def raise_error(e): raise e

        self.statcache = self.metadb.load_statcache()
        result = []
        count = 0

        for dirpath, dirnames, filenames in \
                os.walk(self.prefix, onerror=raise_error):

            # Convert from absolute to relative path.
            assert(dirpath.startswith(self.prefix))
            dirpath = dirpath[len(self.prefix)+1:]

            # TODO: fill stat() cache for directories
            #if self._unmodified(dirpath):
            #    continue

            # Are we in an Attic directory? Then we must check each
            # filename for a "zombie" copy in the parent directory
            # and decide which one to use.
            in_attic = os.path.basename(dirpath) == 'Attic'
            if in_attic:
                parent = os.path.dirname(dirpath)

            for filename in filenames:
                # Ignore all non-RCS files.
                if not filename.endswith(',v'):
                    continue

                count += 1
                progress(_('Collecting RCS files'), count)

                #
                # Perform the zombie check:
                #
                # 1.) If the zombie is in the parent directory, remove
                #     it from 'result' and add the one in the Attic.
                #
                # 2.) If the zombie is in the Attic, leave the good one
                #     in 'result' and ignore the one in the Attic.
                #
                # 3.) If neither of the two files can be classified as
                #     a zombie, raise an error.
                #
                if in_attic and self._zombie_check(result, parent, filename):
                    # This is case 2.) above: skip the Attic filename.
                    continue

                filename = os.path.join(dirpath, filename)
                if not self._unmodified(filename):
                    result.append(filename)

        return result

    def _zombie_check(self, result, parent, filename):
        """Check a path for zombie files.  If a path exists in the Attic
        and the parent directory, one of them must be a zombie copy.  If
        it cannot be determined which one is the zombie and which one is
        the real copy, raise an error.

        This function should only be called during an os.walk() run when
        searching for files an Attic directory.  This guarantees that we
        have already seen the other copy in the parent directory, if one
        exists.  The return value is True if the zombie is in the Attic
        and False if the zombie is in the parent directory."""

        trunkfile = os.path.join(parent, filename)
        if not os.path.isfile(trunkfile):
            # No zombie present; the file exists only in Attic.
            return False

        atticfile = os.path.join(parent, 'Attic', filename)
        # FIXME: Not a reliable test. We should make sure that the
        # zombie contains a subset of the revisions of the real copy.
        if os.path.getsize(trunkfile) < os.path.getsize(atticfile):
            result.remove(trunkfile)
            return False

        raise RuntimeError, \
            _("invalid path: %s (%s)") % (trunkfile, \
            _('exists in Attic and parent directory'))

    def fetch_changes(self, progress=None):
        """Fetch new revisions from the CVS repository.
        """
        if progress == None:
            progress = NoProgress()

        filenames = self.changed_rcs_filenames(progress=progress)
        with progress:
            self._fetch_changes(filenames, progress)

    def _fetch_changes(self, filenames, progress):
        count = 0
        total = len(filenames)
        progress(_('Parsing RCS files'), count, total)

        # We will commit changes to the database every few seconds to
        # avoid having to scan all RCS files again in case the process
        # is interrupted or an error occurs (a rescan isn't really bad
        # but costs time.)
        commit_time = 0
        commit_interval = 10
        self.metadb.begin_transaction()
        for rcsfile in filenames:
            try:
                self._fetch_changes_from_rcsfile(rcsfile)

                if time.time() - commit_time >= commit_interval:
                    try:
                        self.metadb.end_transaction()
                    except:
                        pass
                    self.metadb.begin_transaction()
                    commit_time = time.time()

                count += 1
                progress(_('Parsing RCS files'), count, total)
            except KeyboardInterrupt:
                # Re-raise the exception silently.  An impatient user
                # may interrupt the process and that should by handled
                # gracefully.
                raise
            except:
                # Print the file name where this error happened,
                # regardless of whether the error is actually printed,
                # just as a quick & dirty debugging aid.
                # TODO: raise a FetchChangesError
                print "(Error while processing %s)" % rcsfile
                raise
            finally:
                try:
                    self.metadb.end_transaction()
                except:
                    pass

    def _fetch_changes_from_rcsfile(self, rcsfile):
        # For the working copy path it does not matter if the RCS
        # file is in the 'Attic' directory or not, so strip it.
        filename = self._rcs_strip_attic_re.sub('\\2', rcsfile)

        abspath = os.path.join(self.prefix, rcsfile)
        st = os.stat(abspath)
        identity = (st.st_mtime, st.st_size,)

        for change in RCSFile(abspath).changes():
            # Record the file's actual working copy path, which
            # RCS alone cannot know about.
            change.filename = filename
            self.metadb.add_change(change)

        self.metadb.update_statcache({rcsfile:identity})

    def generate_changesets(self, progress=None):
        """Convert changes stored in the meta database into sets of
        related changes and store the resulting changesets in the meta
        database as well.
        """
        if progress == None:
            progress = NoProgress()

        with progress:
            count = 0
            csg = ChangeSetGenerator()
            total = self.metadb.count_changes()
            progress(_('Processing changes'), 0, total)
            for change in self.changes(processed=False, reentrant=True):
                count += 1
                progress(_('Processing changes'), count, total)
                for cs in csg.integrate(change):
                    self.metadb.add_changeset(cs)
            for cs in csg.finalize():
                self.metadb.add_changeset(cs)

    def fetch(self, progress=None):
        """Fetch new revisions and compute changesets.
        """
        self.fetch_changes(progress)
        self.generate_changesets(progress)

    def changesets(self):
        """Yield new changesets computed earlier.
        """
        for changeset in self.metadb.changesets_by_start_time():
            changeset.provider = self
            yield(changeset)

    def changes(self, processed=None, reentrant=True):
        """Yields changes fetched earlier.

        The 'processed' keyword can be set to a boolean value to
        select whether changes which are already included in a
        previously computed changeset are to be considered.  If the
        value is None, then all changes are considered.
        """
        return self.metadb.changes_by_timestamp(processed=processed,
                                                reentrant=reentrant)

    def blob(self, change, changeset):
        """Return the raw binary content of a file at the specified
        revision.
        """
        filename = change.filename + ',v'
        rcsfile = os.path.join(self.prefix, filename)
        if not os.path.isfile(rcsfile):
            rcsfile = os.path.join(self.prefix,
                os.path.dirname(filename), 'Attic',
                os.path.basename(filename))
        if not os.path.isfile(rcsfile):
            raise RuntimeError, _('no RCS file found for %s') % filename

        # cvs has the odd behavior that it favors revision 1.1 over
        # 1.1.1.1 if it searches for a revision by date and the date
        # is after that of the initial revision even by one second.
        if change.revision == '1.1.1.1' and \
                change.timestamp < changeset.timestamp:
            revision = '1.1'
        else:
            revision = change.revision

        blob = RCSFile(rcsfile).blob(revision)
        if change.mode == 'b':
            return blob
        else:
            return self.expand_keywords(blob, change, rcsfile)

    def expand_keywords(self, line, change, rcsfile):
        return self._rcs_keyword_re.sub(
            lambda match: self.expand_keyword_match(match, change, rcsfile),
            line)

    def expand_keyword_match(self, match, change, rcsfile):
        if self.localid and match.group(1) == self.localid:
            timestamp = time.gmtime(change.timestamp)
            return ('$%s: %s %s %s %s %s $' % \
                (self.localid, os.path.basename(rcsfile),
                 change.revision,
                 time.strftime('%Y/%m/%d %H:%M:%S', timestamp),
                 change.author, change.state)).encode('ascii')
        elif match.group(1) == 'Header':
            timestamp = time.gmtime(change.timestamp)
            return ('$Header: %s %s %s %s %s $' % \
                (rcsfile, change.revision,
                 time.strftime('%Y/%m/%d %H:%M:%S', timestamp),
                 change.author, change.state)).encode('ascii')
        elif match.group(1) == 'Mdocdate':
            # This is for OpenBSD.
            timestamp = time.gmtime(change.timestamp)
            mdocdate = time.strftime('%B %e %Y', timestamp)
            mdocdate = mdocdate.replace('  ', ' ') # for %e
            return ('$Mdocdate: %s $' % mdocdate)
        else:
            return match.group(0)

    def mark_changeset(self, changeset):
        """Mark 'changeset' as having been committed to "the other
        VCS"."""

        assert(changeset.mark != None)
        self.metadb.mark_changeset(changeset)

    def count_changesets(self):
        return self.metadb.count_changesets()
<commit_msg>Fixed a regression in picking revision 1.1 vs. 1.1.1.1<commit_after>"""CVS interface for CVSGit."""

import os.path
import re
import time

from subprocess import Popen, PIPE

from cvsgit.changeset import ChangeSetGenerator
from cvsgit.rcs import RCSFile
from cvsgit.i18n import _
from cvsgit.term import NoProgress

def split_cvs_source(dirname):
    """Split <dirname> into CVSROOT and module paths.
    """
    cvsroot = dirname
    module = ''
    while True:
        parent = os.path.dirname(cvsroot)
        if cvsroot == parent:
            raise TypeError, _('not a CVS repository path (%s): %s') \
                % (_('no CVSROOT within nor above'), dirname)
        if os.path.isdir(os.path.join(cvsroot, 'CVSROOT')):
            return (cvsroot, module,)
        if module == '':
            module = os.path.basename(cvsroot)
        else:
            module = os.path.join(os.path.basename(cvsroot), module)
        cvsroot = parent

class CVS(object):
    """Represents a CVS repository.
    """

    def __init__(self, dirname, metadb):
        self.metadb = metadb

        # 'dirname' is a local filesystem path pointing at the root of
        # a CVS repository or at a module within.  If it is a module
        # path, operations will be limited to that module.  Otherwise,
        # the module path will be empty and operations will by default
        # apply to the whole repository.
        #
        # In any case, the repository path is always determined by
        # recursively following the parents of 'dirname' until a a
        # sub-directory named 'CVSROOT' is found.  Repository path and
        # module path are available for reading in the instance
        # attributes 'root' and 'module', respectively.

        if not os.path.isdir(dirname):
            raise TypeError, _('not a CVS repository path (%s): %s') \
                % (_('not even a directory'), dirname)

        # Convert to absolute pathname, so that our logic doesn't
        # break if anyone uses os.chdir() and so that the path
        # traversal below can always assume an absolute path.
        dirname = os.path.abspath(dirname)

        # Split 'dirname' into self.root and self.module and also
        # set self.prefix to the full absolute module path.
        self.root, self.module = split_cvs_source(dirname)
        if self.module == '':
            self.prefix = self.root
        else:
            self.prefix = os.path.join(self.root, self.module)

        self.localid = None
        self.parse_options()

        self.statcache = {}
        self._rcs_keyword_re = re.compile('\$([^$:\r\n]+)[^$\r\n]*\$')
        self._rcs_headerfix_re = re.compile(' ([^ ]+,v) ')
        self._rcs_strip_attic_re = re.compile('(Attic/)?([^/]+),v$')

    def parse_options(self):
        """Extract relevant information from the CVSROOT/options file,
        if it exsits.  At the moment, the only information that is
        used from the file is the custom Id tag keyword 'tag'."""

        filename = os.path.join(self.root, 'CVSROOT', 'options')
        if not os.path.isfile(filename):
            return

        f = file(filename, 'r')
        try:
            for line in f.readlines():
                option, value = line.split('=', 2)
                if option == 'tag':
                    self.localid = value.strip()
        finally:
            f.close()

    # Helper function to check with the statcache and the stat()
    # system call if a file or directory is unmodified.
    def _unmodified(self, path):
        st = os.stat(os.path.join(self.prefix, path))
        identity = (st.st_mtime, st.st_size,)
        return self.statcache.has_key(path) and \
                self.statcache[path] == identity

    def changed_rcs_filenames(self, progress=None):
        """Return the list of RCS filenames which need to be scanned for
        new changes to import.
        """
        if not progress:
            progress = NoProgress()

        with progress:
            return self._changed_rcs_filenames(progress=progress)

    def _changed_rcs_filenames(self, progress=None):
        # Helper function to raise the OSError reported by os.walk().
        def raise_error(e): raise e

        self.statcache = self.metadb.load_statcache()
        result = []
        count = 0

        for dirpath, dirnames, filenames in \
                os.walk(self.prefix, onerror=raise_error):

            # Convert from absolute to relative path.
            assert(dirpath.startswith(self.prefix))
            dirpath = dirpath[len(self.prefix)+1:]

            # TODO: fill stat() cache for directories
            #if self._unmodified(dirpath):
            #    continue

            # Are we in an Attic directory? Then we must check each
            # filename for a "zombie" copy in the parent directory
            # and decide which one to use.
            in_attic = os.path.basename(dirpath) == 'Attic'
            if in_attic:
                parent = os.path.dirname(dirpath)

            for filename in filenames:
                # Ignore all non-RCS files.
                if not filename.endswith(',v'):
                    continue

                count += 1
                progress(_('Collecting RCS files'), count)

                #
                # Perform the zombie check:
                #
                # 1.) If the zombie is in the parent directory, remove
                #     it from 'result' and add the one in the Attic.
                #
                # 2.) If the zombie is in the Attic, leave the good one
                #     in 'result' and ignore the one in the Attic.
                #
                # 3.) If neither of the two files can be classified as
                #     a zombie, raise an error.
                #
                if in_attic and self._zombie_check(result, parent, filename):
                    # This is case 2.) above: skip the Attic filename.
                    continue

                filename = os.path.join(dirpath, filename)
                if not self._unmodified(filename):
                    result.append(filename)

        return result

    def _zombie_check(self, result, parent, filename):
        """Check a path for zombie files.  If a path exists in the Attic
        and the parent directory, one of them must be a zombie copy.  If
        it cannot be determined which one is the zombie and which one is
        the real copy, raise an error.

        This function should only be called during an os.walk() run when
        searching for files an Attic directory.  This guarantees that we
        have already seen the other copy in the parent directory, if one
        exists.  The return value is True if the zombie is in the Attic
        and False if the zombie is in the parent directory."""

        trunkfile = os.path.join(parent, filename)
        if not os.path.isfile(trunkfile):
            # No zombie present; the file exists only in Attic.
            return False

        atticfile = os.path.join(parent, 'Attic', filename)
        # FIXME: Not a reliable test. We should make sure that the
        # zombie contains a subset of the revisions of the real copy.
        if os.path.getsize(trunkfile) < os.path.getsize(atticfile):
            result.remove(trunkfile)
            return False

        raise RuntimeError, \
            _("invalid path: %s (%s)") % (trunkfile, \
            _('exists in Attic and parent directory'))

    def fetch_changes(self, progress=None):
        """Fetch new revisions from the CVS repository.
        """
        if progress == None:
            progress = NoProgress()

        filenames = self.changed_rcs_filenames(progress=progress)
        with progress:
            self._fetch_changes(filenames, progress)

    def _fetch_changes(self, filenames, progress):
        count = 0
        total = len(filenames)
        progress(_('Parsing RCS files'), count, total)

        # We will commit changes to the database every few seconds to
        # avoid having to scan all RCS files again in case the process
        # is interrupted or an error occurs (a rescan isn't really bad
        # but costs time.)
        commit_time = 0
        commit_interval = 10
        self.metadb.begin_transaction()
        for rcsfile in filenames:
            try:
                self._fetch_changes_from_rcsfile(rcsfile)

                if time.time() - commit_time >= commit_interval:
                    try:
                        self.metadb.end_transaction()
                    except:
                        pass
                    self.metadb.begin_transaction()
                    commit_time = time.time()

                count += 1
                progress(_('Parsing RCS files'), count, total)
            except KeyboardInterrupt:
                # Re-raise the exception silently.  An impatient user
                # may interrupt the process and that should by handled
                # gracefully.
                raise
            except:
                # Print the file name where this error happened,
                # regardless of whether the error is actually printed,
                # just as a quick & dirty debugging aid.
                # TODO: raise a FetchChangesError
                print "(Error while processing %s)" % rcsfile
                raise
            finally:
                try:
                    self.metadb.end_transaction()
                except:
                    pass

    def _fetch_changes_from_rcsfile(self, rcsfile):
        # For the working copy path it does not matter if the RCS
        # file is in the 'Attic' directory or not, so strip it.
        filename = self._rcs_strip_attic_re.sub('\\2', rcsfile)

        abspath = os.path.join(self.prefix, rcsfile)
        st = os.stat(abspath)
        identity = (st.st_mtime, st.st_size,)

        for change in RCSFile(abspath).changes():
            # Record the file's actual working copy path, which
            # RCS alone cannot know about.
            change.filename = filename
            self.metadb.add_change(change)

        self.metadb.update_statcache({rcsfile:identity})

    def generate_changesets(self, progress=None):
        """Convert changes stored in the meta database into sets of
        related changes and store the resulting changesets in the meta
        database as well.
        """
        if progress == None:
            progress = NoProgress()

        with progress:
            count = 0
            csg = ChangeSetGenerator()
            total = self.metadb.count_changes()
            progress(_('Processing changes'), 0, total)
            for change in self.changes(processed=False, reentrant=True):
                count += 1
                progress(_('Processing changes'), count, total)
                for cs in csg.integrate(change):
                    self.metadb.add_changeset(cs)
            for cs in csg.finalize():
                self.metadb.add_changeset(cs)

    def fetch(self, progress=None):
        """Fetch new revisions and compute changesets.
        """
        self.fetch_changes(progress)
        self.generate_changesets(progress)

    def changesets(self):
        """Yield new changesets computed earlier.
        """
        for changeset in self.metadb.changesets_by_start_time():
            changeset.provider = self
            yield(changeset)

    def changes(self, processed=None, reentrant=True):
        """Yields changes fetched earlier.

        The 'processed' keyword can be set to a boolean value to
        select whether changes which are already included in a
        previously computed changeset are to be considered.  If the
        value is None, then all changes are considered.
        """
        return self.metadb.changes_by_timestamp(processed=processed,
                                                reentrant=reentrant)

    def blob(self, change, changeset):
        """Return the raw binary content of a file at the specified
        revision.
        """
        filename = change.filename + ',v'
        rcsfile = os.path.join(self.prefix, filename)
        if not os.path.isfile(rcsfile):
            rcsfile = os.path.join(self.prefix,
                os.path.dirname(filename), 'Attic',
                os.path.basename(filename))
        if not os.path.isfile(rcsfile):
            raise RuntimeError, _('no RCS file found for %s') % filename

        # cvs has the odd behavior that it favors revision 1.1 over
        # 1.1.1.1 if it searches for a revision by date and the date
        # is after that of the initial revision even by one second.
        if change.revision == '1.1.1.1' and \
                change.timestamp < changeset.timestamp:
            revision = '1.1'
        else:
            revision = change.revision

        blob = RCSFile(rcsfile).blob(revision)
        if change.mode == 'b':
            return blob
        else:
            return self.expand_keywords(blob, change, rcsfile, revision)

    def expand_keywords(self, line, change, rcsfile, revision):
        return self._rcs_keyword_re.sub(
            lambda match:
                self.expand_keyword_match(match, change, rcsfile, revision),
            line)

    def expand_keyword_match(self, match, change, rcsfile, revision):
        if match.group(1) == 'Id' or \
                (self.localid and match.group(1) == self.localid):
            timestamp = time.gmtime(change.timestamp)
            return ('$%s: %s %s %s %s %s $' % \
                (match.group(1), os.path.basename(rcsfile),
                 revision,
                 time.strftime('%Y/%m/%d %H:%M:%S', timestamp),
                 change.author, change.state)).encode('ascii')
        elif match.group(1) == 'Header':
            timestamp = time.gmtime(change.timestamp)
            return ('$Header: %s %s %s %s %s $' % \
                (rcsfile, revision,
                 time.strftime('%Y/%m/%d %H:%M:%S', timestamp),
                 change.author, change.state)).encode('ascii')
        elif match.group(1) == 'Mdocdate':
            # This is for OpenBSD.
            timestamp = time.gmtime(change.timestamp)
            mdocdate = time.strftime('%B %e %Y', timestamp)
            mdocdate = mdocdate.replace('  ', ' ') # for %e
            return ('$Mdocdate: %s $' % mdocdate)
        else:
            return match.group(0)

    def mark_changeset(self, changeset):
        """Mark 'changeset' as having been committed to "the other
        VCS"."""

        assert(changeset.mark != None)
        self.metadb.mark_changeset(changeset)

    def count_changesets(self):
        return self.metadb.count_changesets()
<|endoftext|>


<commit_before>from kivy.app import App
from kivy.config import Config
from kivy.clock import *
from kivy.base import *

from kivy.config import ConfigParser
from kivy.uix.settings import Settings

from kivy.uix.screenmanager import * # TODO: Remove

import pentai.db.zodb_dict as z_m
import p_screen_manager as ps_m

from intro_screen import *
from menu_screen import *
from ai_player_screen import *
from human_player_screen import *
from setup_screen import *
from settings_screen import *
from games_screen import *
import pente_screen
from popup import *
import my_button

from pentai.db.games_mgr import *
import pentai.db.openings_book as ob_m

from kivy.properties import ObjectProperty

import pentai.ai.alpha_beta as ab_m

import os
import time
import copy as c_m

import demo as d_m
import pentai.db.misc_db as misc_db

class PentAIApp(App):
    game_filename = StringProperty("")

    def __init__(self, *args, **kwargs):
        self.debug = False
        super(PentAIApp, self).__init__(*args, **kwargs)

    def display_error(self, message):
        self.popup = MessagePopup(title='Error', content=Label(text=message, font_size='20sp'), \
                size_hint=(.9, .2),
                timeout_val=4)
        self.popup.open()
        print message

    def return_screen(self, ignored=None):
        if self.pente_screen:
            self.show_pente_screen()
        else:
            self.show_menu_screen()

    def show_settings_screen(self):
        self.root.set_current("Settings")

    def show_pente_screen(self):
        self.root.set_current("Pente")

    def show_games_screen(self, ignored=None):
        self.root.set_current("Games")

    def show_ai_screen(self, ignored=None):
        self.root.set_current("AI")

    def show_human_screen(self, ignored=None):
        self.root.set_current("Human")

    def show_menu_screen(self, ignored=None):
        self.root.set_current("Menu")

    def show_new_game_screen(self):
        self.game_filename = ""
        self.setup_screen.create_game()
        self.root.set_current("Setup")

    def show_demo(self):
        d = d_m.Demo(self, self.setup_screen.size)

        # Intercept all touch events
        self.root.set_demo(d)

        '''
        for db_inst in [self.games_mgr, self.openings_book, misc_db.get_instance()]:
            # Deep copy and save.
            # (Players are in games_mgr)
            orig = db_inst.__dict__
            db_inst.__dict__ = c_m.deepcopy(db_inst.__dict__)
            db_inst._backup = orig
        '''

        d.start()

    def finish_demo(self):
        z_m.abort()

        self.show_menu_screen()
        self.pente_screen = None
        self.root.set_demo(None)
        #Clock.schedule_once(self.restore_from_demo, .1)

    def in_demo_mode(self):
        return self.root.in_demo_mode()

    '''
    def restore_from_demo(self, *ignored):
        for db_inst in [self.games_mgr, self.openings_book, misc_db.get_instance()]:
            # Restore each to backup
            db_inst.__dict__ = db_inst._backup
    '''

    def load_game_file_cb(self, path, filenames):
        f_n = filenames
        try:
            full_path = os.path.join(path, filenames[0])
        except IndexError:
            self.display_error("Please select a game first")
            return
        self.load_game_file(full_path)

    def edit_game(self, game=None):
        if not game is None:
            self.game = game
        self.setup_screen.alter_game(self.game)
        self.root.current = "Setup"

    def load_game_file(self, full_path=None):
        if full_path != None:
            self.game_filename = full_path
        if self.game_filename == "":
            self.game_filename = self.game.autosave_filename
        # TODO: Check file parsed etc.
        self.setup_screen.load_file(self.game_filename)
        # TODO production app should start game here.
        self.root.current = "Setup"

    def start_game(self, game, screen_size, demo=False):
        # TODO: Move this?
        root = self.root
        try:
            prev_game_screen = root.get_screen("Pente")
            if prev_game_screen != None:
                root.remove_widget(prev_game_screen)
        except ScreenManagerException:
            pass

        self.add_screen(pente_screen.PenteScreen,
            'Pente', screen_size=screen_size,
            filename=self.game_filename)

        self.pente_screen = root.get_screen("Pente")
        self.pente_screen.set_live(not demo)
        self.game = game

        # load the game screen
        self.pente_screen.set_game(game)

        self.show_pente_screen()

    def prompt_quit(self):
        msg_str = "Are you sure you want to quit?"
        ConfirmPopup.create_and_open(message=msg_str,
                    action=self.close_confirmed,
                    size_hint=(.8, .2))

    def close_confirmed(self):
        # TODO Send to the current screen for cleanup?
        current_screen = self.root.current_screen
        if current_screen:
            current_screen.on_pre_leave()
            current_screen.on_leave()
        self.stop()

    def hook_keyboard(self, window, key, *ignored_args):
        # This keyboard control is just for my convenience, not on app.
        print "KEY PRESSED: %s" % key
        typing_screens = ["Setup", "AI", "Human"]
        if key == 27:
            # Escape
            if BasePopup.my_active:
                # Cancel any popup
                BasePopup.clear()
            else:
                if self.root.current == "Settings":
                    self.return_screen()
                elif self.root.current == "Pente" and \
                     self.pente_screen.confirmation_in_progress:
                    self.pente_screen.cancel_confirmation()
                else:
                    self.prompt_quit()
            return True

        elif key == 113:
            # 'q'
            if not self.root.current in typing_screens:
                self.prompt_quit()
            return True

        elif key == 114:
            # 'r'
            if not self.root.current in typing_screens:
                pente_screen.reverse_colours()
            return True

        elif key == 13:
            # Enter
            if ConfirmPopup.is_active():
                ConfirmPopup.confirm()
            elif self.root.current == "Pente":
                self.show_menu_screen()
            return True

        elif key == 32:
            # Space
            # Ignore spaces on other pages, could be entering names
            if self.root.current == "Pente":
                if self.game.finished():
                    self.show_games_screen()
                else:
                    # Game in progress, prompt
                    msg_str = "Leave this game?"
                    ConfirmPopup.create_and_open(message=msg_str,
                        action=self.show_games_screen,
                        size_hint=(.6, .2))
                return True

        elif key == 111:
            if self.root.current in ("Load", "Pente"):
                # or any other screen with text input
                self.show_settings_screen()
                return True

        elif key == 100: # 'd'
            # Debug
            if not self.root.current in typing_screens:
                ab_m.debug = not ab_m.debug
                self.debug = ab_m.debug

                import ai_player as aip_m # hack for debugging
                aip_m.set_skip_openings_book(ab_m.debug)
                print "Debug set to %s" % ab_m.debug
                st() # Could help sometimes?
                return True

        elif key == 8: # 'delete'
            if self.root.current == "Games":
                self.games_screen.delete_game()
            return True

        else:
            if key == 115:
                # 's' for setup
                if self.root.current == "Pente":
                    # Go to setup page
                    # Probably not for production?
                    self.edit_game()
                    return True

        return False

    def add_screen(self, scr_cls, scr_name, **kwargs):
        scr = scr_cls(name=scr_name, **kwargs)
        scr.app = self
        scr.config = self.config
        scr.gm = self.games_mgr
        scr.ob = self.openings_book
        scr.pm = self.games_mgr.players_mgr
        self.root.add_widget(scr)

    def build(self):
        import pentai.db.persistent_dict as pd_m
        if not "db" in os.listdir(self.user_data_dir):
            print "Copying db"
            import shutil
            shutil.copytree("db", os.path.join(self.user_data_dir, "db"))
        ini_file = "pentai.ini"
        ini_path = os.path.join(self.user_data_dir, ini_file)
        if not ini_file in os.listdir(self.user_data_dir):
            print "Copying ini"
            import shutil
            shutil.copy(ini_file, ini_path)
        '''
        import shutil
        #print "Deleting db"
        #shutil.rmtree("db", os.path.join(self.user_data_dir, "db"))
        print "Copying db"
        shutil.copytree("db", os.path.join(self.user_data_dir, "db"))
        '''

        root = ps_m.PScreenManager()
        self.root = root

        self.root.random_transition()

        self.game = None

        self.games_mgr = GamesMgr()
        self.openings_book = ob_m.OpeningsBook(self.games_mgr)

        self.add_screen(IntroScreen, "Intro")

        # Assign to self.config so all screens can get at it.
        self.config = ConfigParser()
        self.config.read(ini_path)

        self.audio = a_m.Audio(self.config)

        Clock.schedule_once(self.create_screens, .5)
        return root

    def create_screens(self, ignored):
        root = self.root
        intro_screen = root.get_screen("Intro")

        screens = [(MenuScreen, "Menu"), (SettingsScreen, "Settings"),
                   (SetupScreen, "Setup"), (GamesScreen, "Games"),
                   (AIPlayerScreen, "AI"), (HumanPlayerScreen, "Human"),
                   ]

        for scr_cls, scr_name in screens:
            self.add_screen(scr_cls, scr_name)

        self.menu_screen = root.get_screen("Menu")
        self.setup_screen = root.get_screen("Setup")
        self.settings_screen = root.get_screen("Settings")
        self.games_screen = root.get_screen("Games")
        self.pente_screen = None

        EventLoop.window.bind(on_keyboard=self.hook_keyboard)
        self.popup = None

        # Pass the config to the settings screen to use it for the
        # Kivy Settings editor.
        self.settings_screen.set_config(self.config)

        self.show_menu_screen()

    def set_confirmation_popups(self):
        p_m.ConfirmPopup.bypass = \
            not self.config.getint("PentAI", "confirm_popups")

    def on_pause(self):
        return True

<commit_msg>Return to live games only<commit_after>from kivy.app import App
from kivy.config import Config
from kivy.clock import *
from kivy.base import *

from kivy.config import ConfigParser
from kivy.uix.settings import Settings

from kivy.uix.screenmanager import * # TODO: Remove

import pentai.db.zodb_dict as z_m
import p_screen_manager as ps_m

from intro_screen import *
from menu_screen import *
from ai_player_screen import *
from human_player_screen import *
from setup_screen import *
from settings_screen import *
from games_screen import *
import pente_screen
from popup import *
import my_button

from pentai.db.games_mgr import *
import pentai.db.openings_book as ob_m

from kivy.properties import ObjectProperty

import pentai.ai.alpha_beta as ab_m

import os
import time
import copy as c_m

import demo as d_m
import pentai.db.misc_db as misc_db

class PentAIApp(App):
    game_filename = StringProperty("")

    def __init__(self, *args, **kwargs):
        self.debug = False
        super(PentAIApp, self).__init__(*args, **kwargs)

    def display_error(self, message):
        self.popup = MessagePopup(title='Error', content=Label(text=message, font_size='20sp'), \
                size_hint=(.9, .2),
                timeout_val=4)
        self.popup.open()
        print message

    def return_screen(self, ignored=None):
        if self.pente_screen and self.pente_screen.game \
                and not self.pente_screen.game.finished():
            self.show_pente_screen()
        else:
            self.show_menu_screen()

    def show_settings_screen(self):
        self.root.set_current("Settings")

    def show_pente_screen(self):
        self.root.set_current("Pente")

    def show_games_screen(self, ignored=None):
        self.root.set_current("Games")

    def show_ai_screen(self, ignored=None):
        self.root.set_current("AI")

    def show_human_screen(self, ignored=None):
        self.root.set_current("Human")

    def show_menu_screen(self, ignored=None):
        self.root.set_current("Menu")

    def show_new_game_screen(self):
        self.game_filename = ""
        self.setup_screen.create_game()
        self.root.set_current("Setup")

    def show_demo(self):
        d = d_m.Demo(self, self.setup_screen.size)

        # Intercept all touch events
        self.root.set_demo(d)

        '''
        for db_inst in [self.games_mgr, self.openings_book, misc_db.get_instance()]:
            # Deep copy and save.
            # (Players are in games_mgr)
            orig = db_inst.__dict__
            db_inst.__dict__ = c_m.deepcopy(db_inst.__dict__)
            db_inst._backup = orig
        '''

        d.start()

    def finish_demo(self):
        z_m.abort()

        self.show_menu_screen()
        self.pente_screen = None
        self.root.set_demo(None)
        #Clock.schedule_once(self.restore_from_demo, .1)

    def in_demo_mode(self):
        return self.root.in_demo_mode()

    '''
    def restore_from_demo(self, *ignored):
        for db_inst in [self.games_mgr, self.openings_book, misc_db.get_instance()]:
            # Restore each to backup
            db_inst.__dict__ = db_inst._backup
    '''

    def load_game_file_cb(self, path, filenames):
        f_n = filenames
        try:
            full_path = os.path.join(path, filenames[0])
        except IndexError:
            self.display_error("Please select a game first")
            return
        self.load_game_file(full_path)

    def edit_game(self, game=None):
        if not game is None:
            self.game = game
        self.setup_screen.alter_game(self.game)
        self.root.current = "Setup"

    def load_game_file(self, full_path=None):
        if full_path != None:
            self.game_filename = full_path
        if self.game_filename == "":
            self.game_filename = self.game.autosave_filename
        # TODO: Check file parsed etc.
        self.setup_screen.load_file(self.game_filename)
        # TODO production app should start game here.
        self.root.current = "Setup"

    def start_game(self, game, screen_size, demo=False):
        # TODO: Move this?
        root = self.root
        try:
            prev_game_screen = root.get_screen("Pente")
            if prev_game_screen != None:
                root.remove_widget(prev_game_screen)
        except ScreenManagerException:
            pass

        self.add_screen(pente_screen.PenteScreen,
            'Pente', screen_size=screen_size,
            filename=self.game_filename)

        self.pente_screen = root.get_screen("Pente")
        self.pente_screen.set_live(not demo)
        self.game = game

        # load the game screen
        self.pente_screen.set_game(game)

        self.show_pente_screen()

    def prompt_quit(self):
        msg_str = "Are you sure you want to quit?"
        ConfirmPopup.create_and_open(message=msg_str,
                    action=self.close_confirmed,
                    size_hint=(.8, .2))

    def close_confirmed(self):
        # TODO Send to the current screen for cleanup?
        current_screen = self.root.current_screen
        if current_screen:
            current_screen.on_pre_leave()
            current_screen.on_leave()
        self.stop()

    def hook_keyboard(self, window, key, *ignored_args):
        # This keyboard control is just for my convenience, not on app.
        print "KEY PRESSED: %s" % key
        typing_screens = ["Setup", "AI", "Human"]
        if key == 27:
            # Escape
            if BasePopup.my_active:
                # Cancel any popup
                BasePopup.clear()
            else:
                if self.root.current == "Settings":
                    self.return_screen()
                elif self.root.current == "Pente" and \
                     self.pente_screen.confirmation_in_progress:
                    self.pente_screen.cancel_confirmation()
                else:
                    self.prompt_quit()
            return True

        elif key == 113:
            # 'q'
            if not self.root.current in typing_screens:
                self.prompt_quit()
            return True

        elif key == 114:
            # 'r'
            if not self.root.current in typing_screens:
                pente_screen.reverse_colours()
            return True

        elif key == 13:
            # Enter
            if ConfirmPopup.is_active():
                ConfirmPopup.confirm()
            elif self.root.current == "Pente":
                self.show_menu_screen()
            return True

        elif key == 32:
            # Space
            # Ignore spaces on other pages, could be entering names
            if self.root.current == "Pente":
                if self.game.finished():
                    self.show_games_screen()
                else:
                    # Game in progress, prompt
                    msg_str = "Leave this game?"
                    ConfirmPopup.create_and_open(message=msg_str,
                        action=self.show_games_screen,
                        size_hint=(.6, .2))
                return True

        elif key == 111:
            if self.root.current in ("Load", "Pente"):
                # or any other screen with text input
                self.show_settings_screen()
                return True

        elif key == 100: # 'd'
            # Debug
            if not self.root.current in typing_screens:
                ab_m.debug = not ab_m.debug
                self.debug = ab_m.debug

                import ai_player as aip_m # hack for debugging
                aip_m.set_skip_openings_book(ab_m.debug)
                print "Debug set to %s" % ab_m.debug
                st() # Could help sometimes?
                return True

        elif key == 8: # 'delete'
            if self.root.current == "Games":
                self.games_screen.delete_game()
            return True

        else:
            if key == 115:
                # 's' for setup
                if self.root.current == "Pente":
                    # Go to setup page
                    # Probably not for production?
                    self.edit_game()
                    return True

        return False

    def add_screen(self, scr_cls, scr_name, **kwargs):
        scr = scr_cls(name=scr_name, **kwargs)
        scr.app = self
        scr.config = self.config
        scr.gm = self.games_mgr
        scr.ob = self.openings_book
        scr.pm = self.games_mgr.players_mgr
        self.root.add_widget(scr)

    def build(self):
        # TODO: Untested yet.
        if not "db.fs" in os.listdir(self.user_data_dir):
            print "Copying db"
            import shutil
            dest = self.user_data_dir
            for fn in ["db.fs", "db.fs.index", "db.fs.tmp"]:
                shutil.copy(fn, dest)

        ini_file = "pentai.ini"
        ini_path = os.path.join(self.user_data_dir, ini_file)
        if not ini_file in os.listdir(self.user_data_dir):
            print "Copying ini"
            import shutil
            shutil.copy(ini_file, ini_path)

        root = ps_m.PScreenManager()
        self.root = root

        self.root.random_transition()

        self.game = None

        self.games_mgr = GamesMgr()
        self.openings_book = ob_m.OpeningsBook(self.games_mgr)

        self.add_screen(IntroScreen, "Intro")

        # Assign to self.config so all screens can get at it.
        self.config = ConfigParser()
        self.config.read(ini_path)

        self.audio = a_m.Audio(self.config)

        Clock.schedule_once(self.create_screens, .5)
        return root

    def create_screens(self, ignored):
        root = self.root
        intro_screen = root.get_screen("Intro")

        screens = [(MenuScreen, "Menu"), (SettingsScreen, "Settings"),
                   (SetupScreen, "Setup"), (GamesScreen, "Games"),
                   (AIPlayerScreen, "AI"), (HumanPlayerScreen, "Human"),
                   ]

        for scr_cls, scr_name in screens:
            self.add_screen(scr_cls, scr_name)

        self.menu_screen = root.get_screen("Menu")
        self.setup_screen = root.get_screen("Setup")
        self.settings_screen = root.get_screen("Settings")
        self.games_screen = root.get_screen("Games")
        self.pente_screen = None

        EventLoop.window.bind(on_keyboard=self.hook_keyboard)
        self.popup = None

        # Pass the config to the settings screen to use it for the
        # Kivy Settings editor.
        self.settings_screen.set_config(self.config)

        self.show_menu_screen()

    def set_confirmation_popups(self):
        p_m.ConfirmPopup.bypass = \
            not self.config.getint("PentAI", "confirm_popups")

    def on_pause(self):
        return True

<|endoftext|>


<commit_before>from collections import namedtuple
from optparse import OptionParser
from urlparse import urlparse

import requests
from logger import logger
from requests.exceptions import ConnectionError

from perfrunner.helpers.remote import RemoteHelper
from perfrunner.settings import ClusterSpec

Build = namedtuple('Build',
                   ['arch', 'pkg', 'edition', 'version', 'release', 'build',
                    'toy', 'url'])

class CouchbaseInstaller(object):

    CBFS = 'http://cbfs-ext.hq.couchbase.com/builds/'
    LATEST_BUILDS = 'http://latestbuilds.hq.couchbase.com/'
    SHERLOCK_BUILDS = ''

    def __init__(self, cluster_spec, options):
        self.options = options
        self.remote = RemoteHelper(cluster_spec, None, options.verbose)
        self.cluster_spec = cluster_spec

        arch = self.remote.detect_arch()
        pkg = self.remote.detect_pkg()

        release = None
        build = None
        if options.version:
            release, build = options.version.split('-')
            self.SHERLOCK_BUILDS = 'http://latestbuilds.hq.couchbase.com/couchbase-server/sherlock/{}/'.format(build)
            if options.toy:
                self.SHERLOCK_BUILDS = 'http://latestbuilds.hq.couchbase.com/couchbase-server/toy-{}/{}/'.format(options.toy, build)

        self.build = Build(arch, pkg, options.cluster_edition, options.version,
                           release, build, options.toy, options.url)
        logger.info('Target build info: {}'.format(self.build))

    def get_expected_filenames(self):
        patterns = ()  # Sentinel
        if self.build.toy:
            patterns = (
                'couchbase-server-community_toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_toy-{toy}-{version}-toy_{arch}.{pkg}',
                'couchbase-server-community_cent58-2.5.2-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent58-3.0.0-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent64-3.0.0-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent64-3.0.1-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent58-master-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent54-master-toy-{toy}-{arch}_{version}-toy.{pkg}',
                # For toy builds >= Sherlock
                'couchbase-server-enterprise-{version}-centos6.{arch}.{pkg}',
            )
        elif self.build.pkg == 'rpm':
            patterns = (
                'couchbase-server-{edition}_centos6_{arch}_{version}-rel.{pkg}',
                'couchbase-server-{edition}-{version}-centos6.{arch}.{pkg}',
                'couchbase-server-{edition}_{arch}_{version}-rel.{pkg}',
            )
        elif self.build.pkg == 'deb':
            patterns = (
                'couchbase-server-{edition}_ubuntu_1204_{arch}_{version}-rel.{pkg}',
                'couchbase-server-{edition}_{version}-ubuntu12.04_amd64.{pkg}',
                'couchbase-server-{edition}_{arch}_{version}-rel.{pkg}',
            )
        elif self.build.pkg == 'exe':
            patterns = (
                'couchbase-server-{edition}_{arch}_{version}-rel.setup.{pkg}',
                'couchbase_server-{edition}-windows-amd64-{version}.{pkg}',
                'couchbase-server-{edition}_{version}-windows_amd64.{pkg}',
                'couchbase_server/{release}/{build}/couchbase_server-{edition}-windows-amd64-{version}.exe',
                'couchbase-server-{edition}_{version}-windows_amd64.{pkg}',
            )

        for pattern in patterns:
            yield pattern.format(**self.build._asdict())

    def find_package(self):
        for filename in self.get_expected_filenames():
            for base in (self.LATEST_BUILDS, self.SHERLOCK_BUILDS, self.CBFS):
                url = '{}{}'.format(base, filename)
                try:
                    status_code = requests.head(url).status_code
                except ConnectionError:
                    continue
                else:
                    if status_code == 200:
                        logger.info('Found "{}"'.format(url))
                        return filename, url
        logger.interrupt('Target build not found')

    def kill_processes(self):
        self.remote.kill_processes()

    def uninstall_package(self):
        self.remote.uninstall_couchbase(self.build.pkg)

    def clean_data(self):
        self.remote.clean_data()

    def install_package(self):
        if not self.options.url:
            filename, url = self.find_package()
        else:
            url = self.options.url
            logger.info("Using this URL to install instead of searching amongst"
                        " the known locations: {}".format(url))
            # obtain the filename after the last '/' of a url.
            filename = urlparse(url).path.split('/')[-1]

        self.remote.install_couchbase(self.build.pkg, url, filename,
                                      self.build.release)

    def install(self):
        self.kill_processes()
        self.uninstall_package()
        self.clean_data()
        self.install_package()

def main():
    usage = '%prog -c cluster -v version [-t toy]'

    parser = OptionParser(usage)

    parser.add_option('-c', dest='cluster_spec_fname',
                      help='path to cluster specification file',
                      metavar='cluster.spec')
    parser.add_option('-e', dest='cluster_edition', default='enterprise',
                      help='the cluster edition (community or enterprise)')
    parser.add_option('-v', dest='version',
                      help='build version', metavar='2.0.0-1976')
    parser.add_option('-t', dest='toy',
                      help='optional toy build ID', metavar='couchstore')
    parser.add_option('--url', dest='url', default=None,
                      help='The http URL to a Couchbase RPM that should be'
                      ' installed. This overrides the URL to be installed.')
    parser.add_option('--verbose', dest='verbose', action='store_true',
                      help='enable verbose logging')

    options, args = parser.parse_args()

    if not (options.cluster_spec_fname and options.version) and not options.url:
        parser.error(
            'Missing mandatory parameter. Either specify both cluster'
            ' spec and version, or specify just the URL to be installed')

    if options.cluster_edition not in ['community', 'enterprise']:
        parser.error('Cluster edition must be either community or enterprise')

    cluster_spec = ClusterSpec()
    cluster_spec.parse(options.cluster_spec_fname, args)

    logger.info ('Install Couchbase {} with options {}'.format(cluster_spec, options))
    installer = CouchbaseInstaller(cluster_spec, options)
    installer.install()

if __name__ == '__main__':
    main()
<commit_msg>add Watson build url<commit_after>from collections import namedtuple
from optparse import OptionParser
from urlparse import urlparse

import requests
from logger import logger
from requests.exceptions import ConnectionError

from perfrunner.helpers.remote import RemoteHelper
from perfrunner.settings import ClusterSpec

Build = namedtuple('Build',
                   ['arch', 'pkg', 'edition', 'version', 'release', 'build',
                    'toy', 'url'])

class CouchbaseInstaller(object):

    CBFS = 'http://cbfs-ext.hq.couchbase.com/builds/'
    LATEST_BUILDS = 'http://latestbuilds.hq.couchbase.com/'
    SHERLOCK_BUILDS = ''
    WATSON_BUILDS = ''

    def __init__(self, cluster_spec, options):
        self.options = options
        self.remote = RemoteHelper(cluster_spec, None, options.verbose)
        self.cluster_spec = cluster_spec

        arch = self.remote.detect_arch()
        pkg = self.remote.detect_pkg()

        release = None
        build = None
        if options.version:
            release, build = options.version.split('-')
            self.SHERLOCK_BUILDS = 'http://latestbuilds.hq.couchbase.com/couchbase-server/sherlock/{}/'.format(build)
            self.WATSON_BUILDS = 'http://172.23.120.24/builds/latestbuilds/couchbase-server/watson/{}/'.format(build)

            if options.toy:
                self.SHERLOCK_BUILDS = 'http://latestbuilds.hq.couchbase.com/couchbase-server/toy-{}/{}/'.format(options.toy, build)

        self.build = Build(arch, pkg, options.cluster_edition, options.version,
                           release, build, options.toy, options.url)
        logger.info('Target build info: {}'.format(self.build))

    def get_expected_filenames(self):
        patterns = ()  # Sentinel
        if self.build.toy:
            patterns = (
                'couchbase-server-community_toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_toy-{toy}-{version}-toy_{arch}.{pkg}',
                'couchbase-server-community_cent58-2.5.2-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent58-3.0.0-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent64-3.0.0-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent64-3.0.1-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent58-master-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent54-master-toy-{toy}-{arch}_{version}-toy.{pkg}',
                # For toy builds >= Sherlock
                'couchbase-server-enterprise-{version}-centos6.{arch}.{pkg}',
            )
        elif self.build.pkg == 'rpm':
            patterns = (
                'couchbase-server-{edition}_centos6_{arch}_{version}-rel.{pkg}',
                'couchbase-server-{edition}-{version}-centos6.{arch}.{pkg}',
                'couchbase-server-{edition}_{arch}_{version}-rel.{pkg}',
                'couchbase-server-{edition}_{version}-{arch}.{pkg}',
            )
        elif self.build.pkg == 'deb':
            patterns = (
                'couchbase-server-{edition}_ubuntu_1204_{arch}_{version}-rel.{pkg}',
                'couchbase-server-{edition}_{version}-ubuntu12.04_amd64.{pkg}',
                'couchbase-server-{edition}_{arch}_{version}-rel.{pkg}',
                'couchbase-server-{edition}_{version}-{arch}.{pkg}',
            )
        elif self.build.pkg == 'exe':
            patterns = (
                'couchbase-server-{edition}_{arch}_{version}-rel.setup.{pkg}',
                'couchbase_server-{edition}-windows-amd64-{version}.{pkg}',
                'couchbase-server-{edition}_{version}-windows_amd64.{pkg}',
                'couchbase_server/{release}/{build}/couchbase_server-{edition}-windows-amd64-{version}.exe',
                'couchbase-server-{edition}_{version}-windows_amd64.{pkg}',
            )

        for pattern in patterns:
            yield pattern.format(**self.build._asdict())

    def find_package(self):
        for filename in self.get_expected_filenames():
            for base in (self.LATEST_BUILDS, self.SHERLOCK_BUILDS, self.WATSON_BUILDS, self.CBFS):
                url = '{}{}'.format(base, filename)
                try:
                    status_code = requests.head(url).status_code
                except ConnectionError:
                    continue
                else:
                    if status_code == 200:
                        logger.info('Found "{}"'.format(url))
                        return filename, url
        logger.interrupt('Target build not found')

    def kill_processes(self):
        self.remote.kill_processes()

    def uninstall_package(self):
        self.remote.uninstall_couchbase(self.build.pkg)

    def clean_data(self):
        self.remote.clean_data()

    def install_package(self):
        if not self.options.url:
            filename, url = self.find_package()
        else:
            url = self.options.url
            logger.info("Using this URL to install instead of searching amongst"
                        " the known locations: {}".format(url))
            # obtain the filename after the last '/' of a url.
            filename = urlparse(url).path.split('/')[-1]

        self.remote.install_couchbase(self.build.pkg, url, filename,
                                      self.build.release)

    def install(self):
        self.kill_processes()
        self.uninstall_package()
        self.clean_data()
        self.install_package()

def main():
    usage = '%prog -c cluster -v version [-t toy]'

    parser = OptionParser(usage)

    parser.add_option('-c', dest='cluster_spec_fname',
                      help='path to cluster specification file',
                      metavar='cluster.spec')
    parser.add_option('-e', dest='cluster_edition', default='enterprise',
                      help='the cluster edition (community or enterprise)')
    parser.add_option('-v', dest='version',
                      help='build version', metavar='2.0.0-1976')
    parser.add_option('-t', dest='toy',
                      help='optional toy build ID', metavar='couchstore')
    parser.add_option('--url', dest='url', default=None,
                      help='The http URL to a Couchbase RPM that should be'
                      ' installed. This overrides the URL to be installed.')
    parser.add_option('--verbose', dest='verbose', action='store_true',
                      help='enable verbose logging')

    options, args = parser.parse_args()

    if not (options.cluster_spec_fname and options.version) and not options.url:
        parser.error(
            'Missing mandatory parameter. Either specify both cluster'
            ' spec and version, or specify just the URL to be installed')

    if options.cluster_edition not in ['community', 'enterprise']:
        parser.error('Cluster edition must be either community or enterprise')

    cluster_spec = ClusterSpec()
    cluster_spec.parse(options.cluster_spec_fname, args)

    installer = CouchbaseInstaller(cluster_spec, options)
    installer.install()

if __name__ == '__main__':
    main()
<|endoftext|>


<commit_before>from collections import namedtuple
from optparse import OptionParser

import requests
from logger import logger
from requests.exceptions import ConnectionError

from perfrunner.helpers.remote import RemoteHelper
from perfrunner.settings import ClusterSpec

Build = namedtuple('Build',
                   ['arch', 'pkg', 'version', 'release', 'build', 'toy'])

class CouchbaseInstaller(object):

    CBFS = 'http://cbfs-ext.hq.couchbase.com/builds/'
    LATEST_BUILDS = 'http://latestbuilds.hq.couchbase.com/'
    SHERLOCK_BUILDS = 'http://latestbuilds.hq.couchbase.com/couchbase-server/sherlock/{build}/'

    def __init__(self, cluster_spec, options):
        self.remote = RemoteHelper(cluster_spec, None, options.verbose)
        self.cluster_spec = cluster_spec

        arch = self.remote.detect_arch()
        pkg = self.remote.detect_pkg()
        release, build = options.version.split('-')

        self.build = Build(arch, pkg, options.version, release, build,
                           options.toy)
        logger.info('Target build info: {}'.format(self.build))

    def get_expected_filenames(self):
        patterns = ()  # Sentinel
        if self.build.toy:
            patterns = (
                'couchbase-server-community_toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_toy-{toy}-{version}-toy_{arch}.{pkg}',
                'couchbase-server-community_cent58-2.5.2-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent58-3.0.0-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent64-3.0.0-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent64-3.0.1-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent58-master-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent54-master-toy-{toy}-{arch}_{version}-toy.{pkg}',
            )
        elif self.build.pkg == 'rpm':
            patterns = (
                'couchbase-server-enterprise_centos6_{arch}_{version}-rel.{pkg}',
                'couchbase-server-enterprise-{version}-centos6.{arch}.{pkg}',
                'couchbase-server-enterprise_{arch}_{version}-rel.{pkg}',
            )
        elif self.build.pkg == 'deb':
            patterns = (
                'couchbase-server-enterprise_ubuntu_1204_{arch}_{version}-rel.{pkg}',
                'couchbase-server-enterprise_{arch}_{version}-rel.{pkg}',
            )
        elif self.build.pkg == 'exe':
            patterns = (
                'couchbase-server-enterprise_{arch}_{version}-rel.setup.{pkg}',
                'couchbase_server-enterprise-windows-amd64-{version}.{pkg}',
                'couchbase_server/{release}/{build}/couchbase_server-enterprise-windows-amd64-{version}.exe',
            )

        for pattern in patterns:
            yield pattern.format(**self.build._asdict())

    def find_package(self):
        for filename in self.get_expected_filenames():
            for base in (self.LATEST_BUILDS, self.CBFS):
                url = '{}{}'.format(base, filename)
                try:
                    status_code = requests.head(url).status_code
                except ConnectionError:
                    continue
                else:
                    if status_code == 200:
                        logger.info('Found "{}"'.format(url))
                        return filename, url
        logger.interrupt('Target build not found')

    def kill_processes(self):
        self.remote.kill_processes()

    def uninstall_package(self):
        self.remote.uninstall_couchbase(self.build.pkg)

    def clean_data(self):
        self.remote.clean_data()

    def install_package(self):
        filename, url = self.find_package()
        self.remote.install_couchbase(self.build.pkg, url, filename,
                                      self.build.release)

    def install(self):
        self.kill_processes()
        self.uninstall_package()
        self.clean_data()
        self.install_package()

def main():
    usage = '%prog -c cluster -v version [-t toy]'

    parser = OptionParser(usage)

    parser.add_option('-c', dest='cluster_spec_fname',
                      help='path to cluster specification file',
                      metavar='cluster.spec')
    parser.add_option('-v', dest='version',
                      help='build version', metavar='2.0.0-1976')
    parser.add_option('-t', dest='toy',
                      help='optional toy build ID', metavar='couchstore')
    parser.add_option('--verbose', dest='verbose', action='store_true',
                      help='enable verbose logging')

    options, _ = parser.parse_args()

    if not options.cluster_spec_fname or not options.version:
        parser.error('Missing mandatory parameter')

    cluster_spec = ClusterSpec()
    cluster_spec.parse(options.cluster_spec_fname)

    installer = CouchbaseInstaller(cluster_spec, options)
    installer.install()

if __name__ == '__main__':
    main()
<commit_msg>install util changes for sherlock builds<commit_after>from collections import namedtuple
from optparse import OptionParser

import requests
from logger import logger
from requests.exceptions import ConnectionError

from perfrunner.helpers.remote import RemoteHelper
from perfrunner.settings import ClusterSpec

Build = namedtuple('Build',
                   ['arch', 'pkg', 'version', 'release', 'build', 'toy'])

class CouchbaseInstaller(object):

    CBFS = 'http://cbfs-ext.hq.couchbase.com/builds/'
    LATEST_BUILDS = 'http://latestbuilds.hq.couchbase.com/'
    SHERLOCK_BUILDS = ''

    def __init__(self, cluster_spec, options):
        self.remote = RemoteHelper(cluster_spec, None, options.verbose)
        self.cluster_spec = cluster_spec

        arch = self.remote.detect_arch()
        pkg = self.remote.detect_pkg()
        release, build = options.version.split('-')
        self.SHERLOCK_BUILDS = 'http://latestbuilds.hq.couchbase.com/couchbase-server/sherlock/{}/'.format(build)

        self.build = Build(arch, pkg, options.version, release, build,
                           options.toy)
        logger.info('Target build info: {}'.format(self.build))

    def get_expected_filenames(self):
        patterns = ()  # Sentinel
        if self.build.toy:
            patterns = (
                'couchbase-server-community_toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_toy-{toy}-{version}-toy_{arch}.{pkg}',
                'couchbase-server-community_cent58-2.5.2-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent58-3.0.0-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent64-3.0.0-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent64-3.0.1-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent58-master-toy-{toy}-{arch}_{version}-toy.{pkg}',
                'couchbase-server-community_cent54-master-toy-{toy}-{arch}_{version}-toy.{pkg}',
            )
        elif self.build.pkg == 'rpm':
            patterns = (
                'couchbase-server-enterprise_centos6_{arch}_{version}-rel.{pkg}',
                'couchbase-server-enterprise-{version}-centos6.{arch}.{pkg}',
                'couchbase-server-enterprise_{arch}_{version}-rel.{pkg}',
            )
        elif self.build.pkg == 'deb':
            patterns = (
                'couchbase-server-enterprise_ubuntu_1204_{arch}_{version}-rel.{pkg}',
                'couchbase-server-enterprise_{arch}_{version}-rel.{pkg}',
            )
        elif self.build.pkg == 'exe':
            patterns = (
                'couchbase-server-enterprise_{arch}_{version}-rel.setup.{pkg}',
                'couchbase_server-enterprise-windows-amd64-{version}.{pkg}',
                'couchbase_server/{release}/{build}/couchbase_server-enterprise-windows-amd64-{version}.exe',
            )

        for pattern in patterns:
            yield pattern.format(**self.build._asdict())

    def find_package(self):
        for filename in self.get_expected_filenames():
            for base in (self.LATEST_BUILDS,self.SHERLOCK_BUILDS, self.CBFS):
                url = '{}{}'.format(base, filename)
                try:
                    status_code = requests.head(url).status_code
                except ConnectionError:
                    continue
                else:
                    if status_code == 200:
                        logger.info('Found "{}"'.format(url))
                        return filename, url
        logger.interrupt('Target build not found')

    def kill_processes(self):
        self.remote.kill_processes()

    def uninstall_package(self):
        self.remote.uninstall_couchbase(self.build.pkg)

    def clean_data(self):
        self.remote.clean_data()

    def install_package(self):
        filename, url = self.find_package()
        self.remote.install_couchbase(self.build.pkg, url, filename,
                                      self.build.release)

    def install(self):
        self.kill_processes()
        self.uninstall_package()
        self.clean_data()
        self.install_package()

def main():
    usage = '%prog -c cluster -v version [-t toy]'

    parser = OptionParser(usage)

    parser.add_option('-c', dest='cluster_spec_fname',
                      help='path to cluster specification file',
                      metavar='cluster.spec')
    parser.add_option('-v', dest='version',
                      help='build version', metavar='2.0.0-1976')
    parser.add_option('-t', dest='toy',
                      help='optional toy build ID', metavar='couchstore')
    parser.add_option('--verbose', dest='verbose', action='store_true',
                      help='enable verbose logging')

    options, _ = parser.parse_args()

    if not options.cluster_spec_fname or not options.version:
        parser.error('Missing mandatory parameter')

    cluster_spec = ClusterSpec()
    cluster_spec.parse(options.cluster_spec_fname)

    installer = CouchbaseInstaller(cluster_spec, options)
    installer.install()

if __name__ == '__main__':
    main()
<|endoftext|>


<commit_before>import logging
import time
from contextlib import contextmanager
from decorator import decorator
from retrying import Retrying
from functools import wraps

__all__ = ['retry_short', 'retry_long', 'set_retry_init', 'disable_retries', 'retry_generator']

_retry_init = None

def set_retry_init(fn):
    global _retry_init
    _retry_init = fn

def _only_one_attempt(args, kwargs):
    kwargs = dict(kwargs, stop_max_attempt_number=0)
    return args, kwargs

def disable_retries():
    set_retry_init(_only_one_attempt)

def reenable_retries():
    set_retry_init(None)

@contextmanager
def disabled_retries():
    disable_retries()
    try:
        yield
    finally:
        reenable_retries()

@decorator
def _warn_about_exceptions(f, *args, **kw):
    try:
        return f(*args, **kw)
    except Exception as e:
        logging.warning("Retrying: {} (message was: {})".format(
            f.__name__, str(e)))
        raise

def initialized_retry(*dargs, **dkw):
    def wrap(f):
        @wraps(f)
        def wrapped_f(*args, **kw):
            if _retry_init:
                rargs, rkw = _retry_init(dargs, dkw)
            else:
                rargs, rkw = dargs, dkw
            return Retrying(*rargs, **rkw).call(_warn_about_exceptions(f), *args, **kw)

        return wrapped_f

    return wrap

# for operations that shouldn't take longer than a few seconds (e.g. HTTP request)
# will retry after 2s, 4s, 8s, 10s, 10s, 10s ... until the 10th attempt
retry_short = initialized_retry(
    wait_exponential_multiplier=1000,
    wait_exponential_max=10000,
    stop_max_attempt_number=10,
)

# for operations that take minutes to finish (e.g. uploading a file)
# will retry after 10s, 20s, 40s, 80s, 2m, ~5m, ~10m, ~20m and then give up.
retry_long = initialized_retry(
    wait_exponential_multiplier=5000,
    stop_max_attempt_number=8,
)

def retry_generator(fn, max_retries=8, retry_multiplier=5.0):
    """
    Retry a generator. The if you don't expect already yielded items to be
    yielded again, the generator will need to keep state about what items have
    already been successfully yielded.
    """
    @wraps(fn)
    def wrapper(*args, **kwargs):
        for retry in range(max_retries):
            try:
                generator = fn(*args, **kwargs)
                for i in generator:
                    yield i
            except StopIteration:
                raise
            except Exception as e:
                logging.warning("Retrying: {} (message was: {})".format(
                    fn.__name__, str(e)))
                time.sleep((retry+1)*retry_multiplier)
            else:
                break
    return wrapper
<commit_msg>Fix decorator parameters<commit_after>import logging
import time
from contextlib import contextmanager
from decorator import decorator
from retrying import Retrying
from functools import wraps

__all__ = ['retry_short', 'retry_long', 'set_retry_init', 'disable_retries', 'retry_generator']

_retry_init = None

def set_retry_init(fn):
    global _retry_init
    _retry_init = fn

def _only_one_attempt(args, kwargs):
    kwargs = dict(kwargs, stop_max_attempt_number=0)
    return args, kwargs

def disable_retries():
    set_retry_init(_only_one_attempt)

def reenable_retries():
    set_retry_init(None)

@contextmanager
def disabled_retries():
    disable_retries()
    try:
        yield
    finally:
        reenable_retries()

@decorator
def _warn_about_exceptions(f, *args, **kw):
    try:
        return f(*args, **kw)
    except Exception as e:
        logging.warning("Retrying: {} (message was: {})".format(
            f.__name__, str(e)))
        raise

def initialized_retry(*dargs, **dkw):
    def wrap(f):
        @wraps(f)
        def wrapped_f(*args, **kw):
            if _retry_init:
                rargs, rkw = _retry_init(dargs, dkw)
            else:
                rargs, rkw = dargs, dkw
            return Retrying(*rargs, **rkw).call(_warn_about_exceptions(f), *args, **kw)

        return wrapped_f

    return wrap

# for operations that shouldn't take longer than a few seconds (e.g. HTTP request)
# will retry after 2s, 4s, 8s, 10s, 10s, 10s ... until the 10th attempt
retry_short = initialized_retry(
    wait_exponential_multiplier=1000,
    wait_exponential_max=10000,
    stop_max_attempt_number=10,
)

# for operations that take minutes to finish (e.g. uploading a file)
# will retry after 10s, 20s, 40s, 80s, 2m, ~5m, ~10m, ~20m and then give up.
retry_long = initialized_retry(
    wait_exponential_multiplier=5000,
    stop_max_attempt_number=8,
)

def retry_generator(fn=None, max_retries=8, retry_multiplier=5.0, *args, **kwargs):
    @decorator
    def _decor_(fn, *args, **kwargs):
        """
        Retry a generator. The if you don't expect already yielded items to be
        yielded again, the generator will need to keep state about what items have
        already been successfully yielded.
        """
        for retry in range(1, max_retries + 1):
            try:
                generator = fn(*args, **kwargs)
                for i in generator:
                    yield i
            except StopIteration:
                raise
            except Exception as e:
                if retry < max_retries:
                    logging.warning("Retrying: {} (message was: {})".format(
                        fn.__name__, str(e)))
                    time.sleep(retry * retry_multiplier)
                else:
                    raise
            else:
                break

    return _decor_(fn, *args, **kwargs) if fn is not None else _decor_
<|endoftext|>


<commit_before>#
# This file is part of my.gpodder.org.
#
# my.gpodder.org is free software: you can redistribute it and/or modify it
# under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# my.gpodder.org is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public
# License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with my.gpodder.org. If not, see <http://www.gnu.org/licenses/>.
#

from mygpo.api.basic_auth import require_valid_user
from django.http import HttpResponse, HttpResponseBadRequest, HttpResponseForbidden, Http404
from mygpo.api.models import Device
from mygpo.api.json import JsonResponse
from django.core import serializers

@require_valid_user()
def subscriptions(request, username, device_uid):

    if request.user.username != username:
        return HttpResponseForbidden()

@require_valid_user()
def episodes(request, username):
    pass

@require_valid_user()
def device(request, username, device_uid):
    pass

@require_valid_user()
def devices(request, username):
    pass

<commit_msg>extended advanced api stubs<commit_after>#
# This file is part of my.gpodder.org.
#
# my.gpodder.org is free software: you can redistribute it and/or modify it
# under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# my.gpodder.org is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public
# License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with my.gpodder.org. If not, see <http://www.gnu.org/licenses/>.
#

from mygpo.api.basic_auth import require_valid_user
from django.http import HttpResponse, HttpResponseBadRequest, HttpResponseForbidden, Http404
from mygpo.api.models import Device
from mygpo.api.json import JsonResponse
from django.core import serializers
from time import mktime
from datetime import datetime
import json

@require_valid_user()
def subscriptions(request, username, device_uid):

    if request.user.username != username:
        return HttpResponseForbidden()

    timestamp = mktime(datetime.now().timetuple())

    if request.method == 'GET':

    elif request.method == 'POST':
        actions = json.loads(request.POST['data'])

    else
        return HttpResponseBadRequest()

@require_valid_user()
def episodes(request, username):
    pass

@require_valid_user()
def device(request, username, device_uid):
    pass

@require_valid_user()
def devices(request, username):
    pass

<|endoftext|>