ws.db.grabbers.recentchanges module

class ws.db.grabbers.recentchanges.GrabberRecentChanges(api, db)

Bases: ws.db.grabbers.GrabberBase.GrabberBase

INSERT_PREDELETE_TABLES = ['recentchanges']

gen_inserts_from_rc(rc)

gen_updates_from_le(logevent)

gen_insert()

A generator for database entries which assumes that the tables are empty.

To be implemented in subclasses.

The yielded values are passed to sqlalchemy.engine.Connection.execute(), so the accepted formats for the values are:

• (stmt, entry) tuple, where stmt is an object accepted by sqlalchemy’s execute method (e.g. plain string or an executable SQL statement construct) and entry is a dict holding the bound parameter values to be used in the execution. The execution is deferred with ws.db.execution.DeferrableExecutionQueue to exploit the executemany execution strategy.

• Or it can yield stmt objects directly, if the executemany execution strategy is not applicable.

needs_update()

Returns True iff there are some recent changes to be fetched from the wiki.

gen_update(since)

A generator for database entries which assumes incremental updates.

To be implemented in subclasses.

If ws.client.api.ShortRecentChangesError is raised while executing this generator, update() starts from scratch with insert(). The exception should be raised as soon as possible to save unnecessary work.

The yielded values should follow the same rules as the gen_insert() method.