ArangoDB数据导入

1.arangoimp方法

参数解析

全局配置部分(Global configuration)

--backslash-escape

use backslash as the escape character for quotes, used for csv (default: false)

--batch-size

size for individual data batches (in bytes) (default: 16777216)

--collection

collection name (default: "")

--configuration

the configuration file or 'none' (default: "")

--convert

convert the strings 'null', 'false', 'true' and strings containing numbers into non-string types (csv and tsv only) (default: true)

--create-collection

create collection if it does not yet exist (default: false)

--create-collection-type

type of collection if collection is created (edge or document). possible values: "document", "edge" (default: "document")

--file

file name ("-" for STDIN) (default: "")

--from-collection-prefix

_from collection name prefix (will be prepended to all values in '_from') (default: "")

--ignore-missing

ignore missing columns in csv input (default: false)

--on-duplicate

action to perform when a unique key constraint violation occurs. Possible values: ignore, replace, update, error. possible values: "error", "ignore", "replace", "update" (default: "error")

--overwrite

overwrite collection if it exist (WARNING: this will remove any data from the collection) (default: false)

--progress

show progress (default: true)

--quote

quote character(s), used for csv (default: """)

--remove-attribute <string...>

remove an attribute before inserting an attribute into a collection (for csv and tsv only) (default: )

--separator

field separator, used for csv and tsv (default: "")

--skip-lines

number of lines to skip for formats (csv and tsv only) (default: 0)

--threads

Number of parallel import threads. Most useful for the rocksdb engine (default: 2)

--to-collection-prefix

_to collection name prefix (will be prepended to all values in '_to') (default: "")

--translate <string...>

translate an attribute name (use as --translate "from=to", for csv and tsv only) (default: )

--type

type of import file. possible values: "auto", "csv", "json", "jsonl", "tsv" (default: "json")

--version

reports the version and exits (default: false)

Section 'log' (Configure the logging)

--log.color

use colors for TTY logging (default: true)

--log.level <string...>

the global or topic-specific log level (default: "info")

--log.output <string...>

log destination(s) (default: )

--log.role

log server role (default: false)

--log.use-local-time

use local timezone instead of UTC (default: false)

--log.use-microtime

use microtime instead (default: false)

Section 'server' (Configure a connection to the server)

--server.authentication

require authentication credentials when connecting (does not affect the server-side authentication settings) (default: true)

--server.connection-timeout

connection timeout in seconds (default: 5)

--server.database

database name to use when connecting (default: "_system")

--server.endpoint

endpoint to connect to, use 'none' to start without a server (default: "http+tcp://127.0.0.1:8529")

--server.password

password to use when connecting. If not specified and authentication is required, the user will be prompted for a password (default: "")

--server.request-timeout

request timeout in seconds (default: 1200)

--server.username

username to use when connecting (default: "root")

Section 'ssl' (Configure SSL communication)

--ssl.protocol

ssl protocol (1 = SSLv2, 2 = SSLv2 or SSLv3 (negotiated), 3 = SSLv3, 4 = TLSv1, 5 = TLSV1.2). possible values: 1, 2, 3, 4, 5 (default: 5)

Section 'temp' (Configure temporary files)

--temp.path

path for temporary files (default: "")

应用实例

导入节点集合数据

arangoimp --server.endpoint tcp://127.0.0.1:8529 --server.username root --server.password ××× --server.database _system --file test.csv --type csv --create-collection true --create-collection-type document --overwrite true --collection "test"

导入边集合数据

arangoimp --server.endpoint tcp://127.0.0.1:8529 --server.username root --server.password *** --server.database _system --file test.csv --type csv --create-collection true --create-collection-type document --overwrite true --collection "test"

python方法

单条导入

from arango import ArangoClient

# Initialize the ArangoDB client.
client = ArangoClient()

# Connect to "test" database as root user.
db = client.db('test', username='root', password='passwd')

# Get the API wrapper for "students" collection.
students = db.collection('students')

# Create some test documents to play around with.
lola = {'_key': 'lola', 'GPA': 3.5, 'first': 'Lola', 'last': 'Martin'}

# Insert a new document. This returns the document metadata.
metadata = students.insert(lola)

批量数据导入

由于每一次insert就会产生一次数据库连接，当数据规模较大时，一次次插入比较浪费网络资源，这时候就需要使用Transactions了

from arango import ArangoClient

# Initialize the ArangoDB client.
client = ArangoClient()

# Connect to "test" database as root user.
db = client.db('test', username='root', password='passwd')

# Get the API wrapper for "students" collection.
students = db.collection('students')

# Begin a transaction via context manager. This returns an instance of
# TransactionDatabase, a database-level API wrapper tailored specifically
# for executing transactions. The transaction is automatically committed
# when exiting the context. The TransactionDatabase wrapper cannot be
# reused after commit and may be discarded after.
with db.begin_transaction() as txn_db:

    # Child wrappers are also tailored for transactions.
    txn_col = txn_db.collection('students')

    # API execution context is always set to "transaction".
    assert txn_db.context == 'transaction'
    assert txn_col.context == 'transaction'

    # TransactionJob objects are returned instead of results.
    job1 = txn_col.insert({'_key': 'Abby'})
    job2 = txn_col.insert({'_key': 'John'})
    job3 = txn_col.insert({'_key': 'Mary'})

# Upon exiting context, transaction is automatically committed.
assert 'Abby' in students
assert 'John' in students
assert 'Mary' in students

# Retrieve the status of each transaction job.
for job in txn_db.queued_jobs():
    # Status is set to either "pending" (transaction is not committed yet
    # and result is not available) or "done" (transaction is committed and
    # result is available).
    assert job.status() in {'pending', 'done'}

# Retrieve the job results.
metadata = job1.result()
assert metadata['_id'] == 'students/Abby'

metadata = job2.result()
assert metadata['_id'] == 'students/John'

metadata = job3.result()
assert metadata['_id'] == 'students/Mary'

# Transactions can be initiated without using a context manager.
# If return_result parameter is set to False, no jobs are returned.
txn_db = db.begin_transaction(return_result=False)
txn_db.collection('students').insert({'_key': 'Jake'})
txn_db.collection('students').insert({'_key': 'Jill'})

# The commit must be called explicitly.
txn_db.commit()
assert 'Jake' in students
assert 'Jill' in students

参考资料

AranfoDB Document v3.3

python-arango document