cubicweb.dataimport

Package containing various utilities to import data into cubicweb.

Utilities

cubicweb.dataimport.count_lines(stream_or_filename)

cubicweb.dataimport.ucsvreader_pb(stream_or_path, encoding='utf-8', delimiter=', ', quotechar='"', skipfirst=False, withpb=True, skip_empty=True)

same as ucsvreader() but a progress bar is displayed as we iter on rows

cubicweb.dataimport.ucsvreader(stream, encoding='utf-8', delimiter=', ', quotechar='"', skipfirst=False, ignore_errors=False, skip_empty=True)

A csv reader that accepts files with any encoding and outputs unicode strings

if skip_empty (the default), lines without any values specified (only separators) will be skipped. This is useful for Excel exports which may be full of such lines.

cubicweb.dataimport.callfunc_every(func, number, iterable)

yield items of iterable one by one and call function func every number iterations. Always call function func at the end.

Object Stores

class cubicweb.dataimport.RQLObjectStore(cnx)

Bases: cubicweb.dataimport.stores.NullStore

Store that works by making RQL queries, hence with all the cubicweb’s machinery activated.

commit()

Commit the database transaction.

prepare_insert_entity(*args, **kwargs)

Given an entity type, attributes and inlined relations, return the inserted entity’s eid.

prepare_insert_relation(eid_from, rtype, eid_to, **kwargs)

Insert into the database a relation rtype between entities with eids eid_from and eid_to.

prepare_update_entity(etype, eid, **kwargs)

Given an entity type and eid, update the corresponding entity with specified attributes and inlined relations.

rql(*args)

Execute a RQL query. This is NOT part of the store API.

class cubicweb.dataimport.NoHookRQLObjectStore(cnx, metagen=None)

Bases: cubicweb.dataimport.stores.RQLObjectStore

Store that works by accessing low-level CubicWeb’s source API, with all hooks deactivated. It may be given a metadata generator object to handle metadata which are usually handled by hooks.

Arguments: - cnx, a connection to the repository - metagen, optional MetadataGenerator instance

prepare_insert_entity(etype, **kwargs)

Given an entity type, attributes and inlined relations, returns the inserted entity’s eid.

prepare_insert_relation(eid_from, rtype, eid_to, **kwargs)

Insert into the database a relation rtype between entities with eids eid_from and eid_to.