Table of Contents

1. Introduction

2. Installing

1. Linux

2. Mac OS X

3. Windows

4. ARM

5. Compiling

6. Upgrading

1. Incompatible changes in 2.8

2. Upgrading to 2.8

3. Incompatible changes in 2.7

4. Incompatible changes in 2.6

5. Upgrading to 2.6

6. Incompatible changes in 2.5

7. Upgrading to 2.5

8. Incompatible changes in 2.4

9. Upgrading to 2.4

10. Incompatible changes in 2.3

11. Upgrading to 2.3

12. Upgrading to 2.2

7. Cluster setup

3. Whats New

1. Whats New in 2.7

2. Whats New in 2.6

3. Whats New in 2.5

4. Whats New in 2.4

5. Whats New in 2.3

6. Whats New in 2.2

7. Whats New in 2.1

4. First Steps

1. Getting Familiar

2. The ArangoDB Server

3. The ArangoDB Shell

1. Shell Output

2. Configuration

3. Details

4. Collections

5. ArangoDB Web Interface

6. Creating and Dropping Databases

1. Working with Databases

2. Notes about Databases

7. Creating and Dropping Collections

1. Collection Methods

2. Database Methods

8. Working with Documents

1. Address and ETag

2. Collection Methods

3. Database Methods

9. Indexing

1. Index Basics

2. Which Index to use when

3. How ArangoDB uses indexes

4. Working with Indexes

1. Hash Indexes

2. Skiplists

3. Fulltext Indexes

4. Geo Indexes

5. Cap Constraint

10. Simple Queries

1. Sequential Access

2. Pagination

3. Modification Queries

4. Geo Queries

5. Fulltext Queries

11. Transactions

1. Transaction invocation

2. Passing parameters

3. Locking and isolation

4. Durability

5. Limitations

12. AQL

1. How to invoke AQL

2. Data modification queries

3. The AQL query optimizer

4. The AQL query result cache

5. Language Basics

6. Functions

1. Type cast

2. String

3. Numeric

4. Date

5. Array

6. Object / Document

7. Geo

8. Fulltext

9. Miscellaneous

7. Query Results

8. Operators

9. High level Operations

10. Graphs

1. Traversal

2. Named Operations

3. Other

11. Advanced Features

12. Extending AQL

1. Conventions

2. Registering Functions

13. AQL Examples

1. Collection based queries

2. Data-modification queries

3. Projections and filters

4. Joins

5. Grouping

6. Traversals

13. Graphs

1. General Graphs

1. Graph Management

2. Graph Functions

3. Fluent Query Interface

2. Traversals

1. Using Traversal Objects

2. Example Data

3. Working with Edges

14. Foxx

1. In a nutshell

2. Install

1. Github

2. Store

3. Local

4. Remote

5. Generate

3. Development

1. Dev-Mode

2. Debugging

3. Source Files

4. Controller

5. Scripts

6. Model

7. Repository

8. Queries

9. Sessions

10. Background Tasks

11. Console API

12. Meta information

13. Exports

14. Documentation

4. Production

1. Prod-Mode

2. Debugging

3. New Versions

5. Cluster

1. Install

2. Setup & Teardown

15. ArangoDB's Actions

1. Delivering HTML Pages

2. Json Objects

3. Modifying

16. Replication

1. Components

2. Example Setup

3. Syncing Collections

4. Replication Limitations

5. Replication Overhead

17. Sharding

1. How to try it out

2. Implementation

3. Authentication

4. Firewall setup

18. Server Configuration

1. Arangod options

2. Write-ahead log options

3. Managing Endpoints

4. Cluster options

5. Logging options

6. Communication options

7. Authentication

8. Emergency Console

19. HTTP API

1. General HTTP Handling

2. Databases

1. To-Endpoint

2. Management

3. Notes on Databases

3. Documents

1. Address and ETag

2. Working with

4. Edges

1. Address and ETag

2. Working with Edges

5. Exporting data

6. AQL Query Cursors

1. Query Results

2. Accessing Cursors

7. AQL Queries

8. AQL Query Cache

9. AQL User Functions Management

10. Simple Queries

11. Collections

1. Creating

2. Getting Information

3. Modifying

12. Indexes

1. Working with Indexes

2. Cap Constraints

3. Hash

4. Skiplist

5. Geo

6. Fulltext

13. Transactions

14. General Graph

1. Management

2. Vertices

3. Edges

15. Traversals

16. Replication

1. Replication Dump

2. Replication Logger

3. Replication Applier

4. Other Replication Commands

17. Tasks

18. Monitoring

19. User Management

20. Async Result

21. Endpoints

22. Sharding

23. Miscellaneous functions

20. Bulk Import / Export

1. JSON Documents HTTP-API

2. Headers & Values HTTP-API

3. Batch Requests HTTP-API

4. Arangoimp

5. Arangodump

6. Arangorestore

21. JavaScript Modules

1. "console"

2. "fs"

3. "process"

4. "request"

5. "actions"

6. "queries"

7. "planner"

8. Write-ahead log

9. Task Management

10. Using jsUnity

22. Administrating ArangoDB

23. Advanced

1. Arangob

2. Write-ahead log

3. Server Internals

4. Datafile Debugger

24. Naming Conventions

1. Database Names

2. Collection Names

3. Document Keys

4. Attribute Names

25. Error codes and meanings

26. Glossary

ArangoDB v2.8.9 Documentation

Welcome to the ArangoDB documentation!

The documentation introduces ArangoDB for you as a user, developer and administrator and describes all of its functions in detail.

ArangoDB is a multi-model, open-source database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions. Use ACID transactions if you require them. Scale horizontally and vertically with a few mouse clicks.

Key features include:

- Schema-free schemata let you combine the space efficiency of MySQL with the performance power of NoSQL

- Use ArangoDB as an application server and fuse your application and database together for maximal throughput

- JavaScript for all: no language zoo, you can use one language from your browser to your back-end

- ArangoDB is multi-threaded - exploit the power of all your cores

- Flexible data modeling: model your data as combination of key-value pairs, documents or graphs - perfect for social relations

- Free index choice: use the correct index for your problem, be it a skip list or a fulltext search

- Configurable durability: let the application decide if it needs more durability or more performance

- No-nonsense storage: ArangoDB uses all of the power of modern storage hardware, like SSD and large caches

- Powerful query language (AQL) to retrieve and modify data

- Transactions: run queries on multiple documents or collections with optional transactional consistency and isolation

- Replication and Sharding: set up the database in a master-slave configuration or spread bigger datasets across multiple servers

- It is open source (Apache License 2.0)

In this documentation you can inform yourself about all the functions, features and programs ArangoDB provides for you. Features are ilustrated with interactive usage examples; you can cut'n'paste them into arangosh to try them out. The http REST-API is demonstrated with cut'n'paste recepies intended to be used with the cURL. Drivers may provide their own examples based on these .js based examples to improve understandeability for their respective users. I.e. for the java driver some of the samples are re-implemented.

You can also go to our cookbook and look through some recipes to learn more about ArangoDB specific problems and solutions.

Community

If you have questions regarding ArangoDB, Foxx, drivers, or this documentation don't hesitate to contact us on:

- Github for issues and misbehavior or pull requests

- Google groups for discussions about ArangoDB in general or to announce your new Foxx App

- Stackoverflow for questions about AQL, usage scenarios etc.

Please describe:

- the environment you run ArangoDB in

- the ArangoDB version you use

- whether you're using Foxx

- the client you're using

- which parts of the Documentation you're working with (link)

- what you expect to happen

- whats actually happening

We will respond as soon as possible.

Installing

This chapter describes how to install ArangoDB under various operation systems.

First of all download and install the corresponding RPM or Debian package or use homebrew on MacOS X. You can find packages for various operation systems at our install section.

In this chapter you will also learn how to compile ArangoDB from scratch.

You also get help if you want to update your ArangoDB version to the newest one!

Linux

You can find binary packages for various Linux distributions here.

We provide packages for:

- Centos

- Debian

- Fedora

- Linux-Mint

- Mandriva

- OpenSUSE

- RedHat RHEL

- SUSE SLE

- Ubuntu

Using a Package Manager to install ArangoDB

Follow the instructions on the install page to use your favorite package manager for the major distributions. After setting up the ArangoDB repository you can easily install ArangoDB using yum, aptitude, urpmi or zypper.

Linux-Mint

Please use the corresponding Ubuntu or Debian packages.

Using Vagrant and Chef

A Chef recipe is available from jbianquetti at:

https://github.com/jbianquetti/chef-arangodb

Using ansible

An Ansible role is available through Ansible-Galaxy

- Role on Ansible-Galaxy: https://galaxy.ansible.com/list#/roles/2344

- Source on Github: https://github.com/stackmagic/ansible-arangodb

Mac OS X

The preferred method for installing ArangoDB under Mac OS X is homebrew. However, in case you are not using homebrew, we provide a command-line app which contains all the executables.

There is also a version available in the AppStore, which comes with a nice graphical user interface to start and stop the server.

Homebrew

If you are using homebrew, then you can install the ArangoDB using brew as follows:

brew install arangodb

This will install the current stable version of ArangoDB and all dependencies within your Homebrew tree. Note that the server will be installed as:

/usr/local/sbin/arangod

The ArangoDB shell will be installed as:

/usr/local/bin/arangosh

If you want to install the latest (unstable) version use:

brew install --HEAD arangodb

You can uninstall ArangoDB using:

brew uninstall arangodb

However, in case you started ArangoDB using the launchctl, you need to unload it before uninstalling the server:

launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.arangodb.plist

Then remove the LaunchAgent:

rm ~/Library/LaunchAgents/homebrew.mxcl.arangodb.plist

Note: If the latest ArangoDB Version is not shown in homebrew, you also need to update homebrew:

brew update

Known issues

- Performance - the LLVM delivered as of Mac OS X El Capitan builds slow binaries. Use GCC instead

- the Commandline argument parsing doesn't accept blanks in filenames; the CLI version below does.

Apple's App Store

ArangoDB is available in Apple's App-Store. Please note, that it sometimes takes days or weeks until the latest versions are available.

Command-Line App

In case you are not using homebrew, we also provide a command-line app. You can download it from here.

Choose Mac OS X and go to Grab binary packages directly. This allows you to install the application ArangoDB-CLI in your application folder.

Starting the application will start the server and open a terminal window showing you the log-file.

ArangoDB server has been started

The database directory is located at

'/Applications/ArangoDB-CLI.app/Contents/MacOS/opt/arangodb/var/lib/arangodb'

The log file is located at

'/Applications/ArangoDB-CLI.app/Contents/MacOS/opt/arangodb/var/log/arangodb/arangod.log'

You can access the server using a browser at 'http://127.0.0.1:8529/'

or start the ArangoDB shell

'/Applications/ArangoDB-CLI.app/Contents/MacOS/arangosh'

Switching to log-file now, killing this windows will NOT stop the server.

2013-10-27T19:42:04Z [23840] INFO ArangoDB (version 1.4.devel [darwin]) is ready for business. Have fun!

Note that it is possible to install both, the homebrew version and the command-line app. You should, however, edit the configuration files of one version and change the port used.

Windows

The default installation directory is c:\Program Files\ArangoDB-1.x.y. During the installation process you may change this. In the following description we will assume that ArangoDB has been installed in the location <ROOTDIR>.

You have to be careful when choosing an installation directory. You need either write permission to this directory or you need to modify the config file for the server process. In the latter case the database directory and the Foxx directory has to be writable by the user.

Installing for a single user: Select a different directory during installation. For example c:\Users\<Username>\ArangoDB or c:\ArangoDB.

Installing for multiple users: Keep the default directory. After the installation edit the file <ROOTDIR>\etc\ArangoDB\arangod.conf. Adjust the directory and app-path so that these paths point into your home directory.

[database]

directory = @HOMEDRIVE@\@HOMEPATH@\arangodb\databases

[javascript]

app-path = @HOMEDRIVE@\@HOMEPATH@\arangodb\apps

Create the directories for each user that wants to use ArangoDB.

Installing as Service: Keep the default directory. After the installation open a command line as administrator (search for cmd and right click run as administrator).

cmd&gt; arangod --install-service

INFO: adding service 'ArangoDB - the multi-model database' (internal 'ArangoDB')

INFO: added service with command line '"C:\Program Files (x86)\ArangoDB 1.4.4\bin\arangod.exe" --start-service'

Open the service manager and start ArangoDB. In order to enable logging

edit the file "<ROOTDIR>\etc\arangodb\arangod.conf" and uncomment the file

option.

[log]

file = @ROOTDIR@\var\log\arangodb\arangod.log

Client, Server and Lock-Files

Please note that ArangoDB consists of a database server and client tools. If you start the server, it will place a (read-only) lock file to prevent accidental access to the data. The server will attempt to remove this lock file when it is started to see if the lock is still valid - this is in case the installation did not proceed correctly or if the server terminated unexpectedly.

Starting

To start an ArangoDB server instance with networking enabled, use the executable arangod.exe located in <ROOTDIR>\bin. This will use the configuration file arangod.conf located in <ROOTDIR>\etc\arangodb, which you can adjust to your needs and use the data directory <ROOTDIR>\var\lib\arangodb. This is the place where all your data (databases and collections) will be stored by default.

Please check the output of the arangod.exe executable before going on. If the server started successfully, you should see a line  ArangoDB is ready for business. Have fun!  at the end of its output.

We now wish to check that the installation is working correctly and to do this we will be using the administration web interface. Execute arangod.exe if you have not already done so, then open up your web browser and point it to the page:

http://127.0.0.1:8529/

To check if your installation was successful, click the Collection tab and open the configuration. Select the System type. If the installation was successful, then the page should display a few system collections.

Try to add a new collection and then add some documents to this new collection. If you have succeeded in creating a new collection and inserting one or more documents, then your installation is working correctly.

Advanced Starting

If you want to provide our own start scripts, you can set the environment variable ARANGODB_CONFIG_PATH. This variable should point to a directory containing the configuration files.

Using the Client

To connect to an already running ArangoDB server instance, there is a shell arangosh.exe located in <ROOTDIR>\bin. This starts a shell which can be used – amongst other things – to administer and query a local or remote ArangoDB server.

Note that arangosh.exe does NOT start a separate server, it only starts the shell. To use it you must have a server running somewhere, e.g. by using the arangod.exe executable.

arangosh.exe uses configuration from the file arangosh.conf located in <ROOTDIR>\etc\arangodb\. Please adjust this to your needs if you want to use different connection settings etc.

32bit

If you have an EXISTING database, then please note that currently a 32 bit version of ArangoDB is NOT compatible with a 64 bit version. This means that if you have a database created with a 32 bit version of ArangoDB it may become corrupted if you execute a 64 bit version of ArangoDB against the same database, and vice versa.

Upgrading

To upgrade an EXISTING database created with a previous version of ArangoDB, please execute the server arangod.exe with the option --upgrade. Otherwise starting ArangoDB may fail with errors.

Note that there is no harm in running the upgrade. So you should run this batch file if you are unsure of the database version you are using.

You should always check the output for errors to see if the upgrade was completed successfully.

Uninstalling

To uninstall the Arango server application you can use the windows control panel (as you would normally uninstall an application). Note however, that any data files created by the Arango server will remain as well as the <ROOTDIR> directory. To complete the uninstallation process, remove the data files and the <ROOTDIR> directory manually.

Limitations for Cygwin

Please note some important limitations when running ArangoDB under Cygwin: Starting ArangoDB can be started from out of a Cygwin terminal, but pressing CTRL-C will forcefully kill the server process without giving it a chance to handle the kill signal. In this case, a regular server shutdown is not possible, which may leave a file LOCK around in the server's data directory. This file needs to be removed manually to make ArangoDB start again. Additionally, as ArangoDB does not have a chance to handle the kill signal, the server cannot forcefully flush any data to disk on shutdown, leading to potential data loss. When starting ArangoDB from a Cygwin terminal it might also happen that no errors are printed in the terminal output. Starting ArangoDB from an MS-DOS command prompt does not impose these limitations and is thus the preferred method.

Please note that ArangoDB uses UTF-8 as its internal encoding and that the system console must support a UTF-8 codepage (65001) and font. It may be necessary to manually switch the console font to a font that supports UTF-8.

ARM

The ArangoDB packages for ARM require the kernel to allow unaligned memory access. How the kernel handles unaligned memory access is configurable at runtime by checking and adjusting the contents  /proc/cpu/alignment .

In order to operate on ARM, ArangoDB requires the bit 1 to be set. This will make the kernel trap and adjust unaligned memory accesses. If this bit is not set, the kernel may send a SIGBUS signal to ArangoDB and terminate it.

To set bit 1 in  /proc/cpu/alignment  use the following command as a privileged user (e.g. root):

echo "2" > /proc/cpu/alignment

Note that this setting affects all user processes and not just ArangoDB. Setting the alignment with the above command will also not make the setting permanent, so it will be lost after a restart of the system. In order to make the setting permanent, it should be executed during system startup or before starting arangod.

The ArangoDB start/stop scripts do not adjust the alignment setting, but rely on the environment to have the correct alignment setting already. The reason for this is that the alignment settings also affect all other user processes (which ArangoDB is not aware of) and thus may have side-effects outside of ArangoDB. It is therefore more reasonable to have the system administrator carry out the changes.

If the alignment settings are not correct, ArangoDB will log an error at startup and abort.

Compiling ArangoDB from scratch

The following sections describe how to compile and build the ArangoDB from scratch. ArangoDB will compile on most Linux and Mac OS X systems. We assume that you use the GNU C/C++ compiler or clang/clang++ to compile the source. ArangoDB has been tested with the GNU C/C++ compiler and clang/clang++, but should be able to compile with any Posix-compliant, C++11-enabled compiler. Please let us know whether you successfully compiled it with another C/C++ compiler.

By default, cloning the github repository will checkout devel. This version contains the development version of the ArangoDB. Use this branch if you want to make changes to the ArangoDB source.

Devel Version

Note: a separate blog article is available that describes how to compile ArangoDB from source on Ubuntu.

Basic System Requirements

Verify that your system contains

- the GNU C/C++ compilers "gcc" and "g++" and the standard C/C++ libraries, with support for C++11. You will need version gcc 4.9.0 or higher. For "clang" and "clang++", you will need at least version 3.6.

- the GNU autotools (autoconf, automake)

- GNU make

- the GNU scanner generator FLEX, at least version 2.3.35

- the GNU parser generator BISON, at least version 2.4

- Python, version 2 or 3

- the OpenSSL library, version 1.0.1g or higher (development package)

- the GNU readline library (development package)

- Go, at least version 1.4.1

Most Linux systems already supply RPMs or DPKGs for these packages. Some distributions, for example Ubuntu 12.04 or Centos 5, provide only very out-dated versions of compilers, FLEX, BISON, and/or the V8 engine. In that case you need to compile newer versions of the programs and/or libraries.

When compiling with special configure options, you may need the following extra libraries:

- the Boost test framework library (only when using configure option  --enable-maintainer-mode )

Download the Source

Download the latest source using git:

unix> git clone git://github.com/arangodb/arangodb.git

This will automatically clone the devel branch.

Note: if you only plan to compile ArangoDB locally and do not want to modify or push any changes, you can speed up cloning substantially by using the --single-branch and --depth parameters for the clone command as follows:

unix> git clone --single-branch --depth 1 git://github.com/arangodb/arangodb.git

Setup

Switch into the ArangoDB directory

unix> cd ArangoDB

In order to generate the configure script, execute

unix> make setup

This will call aclocal, autoheader, automake, and autoconf in the correct order.

Configure

In order to configure the build environment please execute

unix> ./configure

to setup the makefiles. This will check the various system characteristics and installed libraries.

Please note that it may be required to set the --host and --target variables when running the configure command. For example, if you compile on MacOS, you should add the following options to the configure command:

--host=x86_64-apple-darwin --target=x86_64-apple-darwin

The host and target values for other architectures vary.

If you also plan to make changes to the source code of ArangoDB, add the following option to the configure command: --enable-maintainer-mode. Using this option, you can make changes to the lexer and parser files and some other source files that will generate other files. Enabling this option will add extra dependencies to BISON, FLEX, and PYTHON. These external tools then need to be available in the correct versions on your system.

The following configuration options exist:

--enable-relative

This will make relative paths be used in the compiled binaries and scripts. It allows to run ArangoDB from the compile directory directly, without the need for a make install command and specifying much configuration parameters. When used, you can start ArangoDB using this command:

bin/arangod /tmp/database-dir

ArangoDB will then automatically use the configuration from file etc/relative/arangod.conf.

--enable-all-in-one-etcd

This tells the build system to use the bundled version of ETCD. This is the default and recommended.

--enable-internal-go

This tells the build system to use Go binaries located in the 3rdParty directory. Note that ArangoDB does not ship with Go binaries, and that the Go binaries must be copied into this directory manually.

--enable-maintainer-mode

This tells the build system to use BISON and FLEX to regenerate the parser and scanner files. If disabled, the supplied files will be used so you cannot make changes to the parser and scanner files. You need at least BISON 2.4.1 and FLEX 2.5.35. This option also allows you to make changes to the error messages file, which is converted to js and C header files using Python. You will need Python 2 or 3 for this. Furthermore, this option enables additional test cases to be executed in a make unittests run. You also need to install the Boost test framework for this.

Additionally, turning on the maintainer mode will turn on a lot of assertions in the code.

--enable-failure-tests

This option activates additional code in the server that intentionally makes the server crash or misbehave (e.g. by pretending the system ran out of memory). This option is useful for writing tests.

--enable-v8-debug

Builds a debug version of the V8 library. This is useful only when working on the V8 integration inside ArangoDB.

--enable-tcmalloc

Links arangod and the client tools against the tcmalloc library installed on the system. Note that when this option is set, a tcmalloc library must be present and exposed under the name  libtcmalloc ,  libtcmalloc_minimal  or  libtcmalloc_debug .

Compiling Go

Users F21 and duralog told us that some systems don't provide an update-to-date version of go. This seems to be the case for at least Ubuntu 12 and 13. To install go on these system, you may follow the instructions provided here. For other systems, you may follow the instructions here.

To make ArangoDB use a specific version of go, you may copy the go binaries into the 3rdParty/go-32 or 3rdParty/go-64 directories of ArangoDB (depending on your architecture), and then tell ArangoDB to use this specific go version by using the --enable-internal-go configure option.

User duralog provided some the following script to pull the latest release version of go into the ArangoDB source directory and build it:

cd ArangoDB

hg clone -u release https://code.google.com/p/go 3rdParty/go-64 && \

cd 3rdParty/go-64/src && \

./all.bash

# now that go is installed, run your configure with --enable-internal-go

./configure --enable-internal-go

Compile

Compile the programs (server, client, utilities) by executing

make

This will compile ArangoDB and create a binary of the server in

./bin/arangod

Test

Create an empty directory

unix> mkdir /tmp/database-dir

Check the binary by starting it using the command line.

unix> ./bin/arangod -c etc/relative/arangod.conf --server.endpoint tcp://127.0.0.1:8529 /tmp/database-dir

This will start up the ArangoDB and listen for HTTP requests on port 8529 bound to IP address 127.0.0.1. You should see the startup messages similar to the following:

2013-10-14T12:47:29Z [29266] INFO ArangoDB xxx ...

2013-10-14T12:47:29Z [29266] INFO using endpoint 'tcp://127.0.0.1.8529' for non-encrypted requests

2013-10-14T12:47:30Z [29266] INFO Authentication is turned off

2013-10-14T12:47:30Z [29266] INFO ArangoDB (version xxx) is ready for business. Have fun!

If it fails with a message about the database directory, please make sure the database directory you specified exists and can be written into.

Use your favorite browser to access the URL

http://127.0.0.1:8529/_api/version

This should produce a JSON object like

{"server" : "arango", "version" : "..."}

as result.

Re-building ArangoDB after an update

To stay up-to-date with changes made in the main ArangoDB repository, you will need to pull the changes from it and re-run  make .

Normally, this will be as simple as follows:

unix> git pull

unix> make

From time to time there will be bigger structural changes in ArangoDB, which may render the old Makefiles invalid. Should this be the case and  make  complains about missing files etc., the following commands should fix it:

unix> rm -rf lib/*/.deps arangod/*/.deps arangosh/*/.deps Makefile

unix> make setup

unix> ./configure <your configure options go here>

unix> make

In order to reset everything and also recompile all 3rd party libraries, issue the following commands:

unix> make superclean

unix> git checkout -- .

unix> make setup

unix> ./configure <your configure options go here>

unix> make

This will clean up ArangoDB and the 3rd party libraries, and rebuild everything.

If you forgot your previous configure options, you can look them up with

unix> head config.log

before issuing  make superclean  (as make  superclean  also removes the file  config.log ).

Sometimes you can get away with the less intrusive commands.

Install

Install everything by executing

make install

You must be root to do this or at least have write permission to the corresponding directories.

The server will by default be installed in

/usr/local/sbin/arangod

The configuration file will be installed in

/usr/local/etc/arangodb/arangod.conf

The database will be installed in

/usr/local/var/lib/arangodb

The ArangoShell will be installed in

/usr/local/bin/arangosh

Note: The installation directory will be different if you use one of the  precompiled  packages. Please check the default locations of your operating system, e. g.  /etc  and  /var/lib .

When upgrading from a previous version of ArangoDB, please make sure you inspect ArangoDB's log file after an upgrade. It may also be necessary to start ArangoDB with the --upgrade parameter once to perform required upgrade or initialization tasks.

General Upgrade Information

Recommended upgrade procedure

To upgrade an existing ArangoDB database to a newer version of ArangoDB (e.g. 2.2 to 2.3, or 2.3 to 2.4), the following method is recommended:

- Check the CHANGELOG and the list of incompatible changes for API or other changes in the new version of ArangoDB and make sure your applications can deal with them

- Stop the "old" arangod service or binary

- Copy the entire "old" data directory to a safe place (that is, a backup)

- Install the new version of ArangoDB and start the server with the --upgrade option once. This might write to the logfile of ArangoDB, so you may want to check the logs for any issues before going on.

- Start the "new" arangod service or binary regularly and check the logs for any issues. When you're confident everything went well, you may want to check the database directory for any files with the ending .old. These files are created by ArangoDB during upgrades and can be safely removed manually later.

If anything goes wrong during or shortly after the upgrade:

- Stop the "new" arangod service or binary

- Revert to the "old" arangod binary and restore the "old" data directory

- Start the "old" version again

It is not supported to use datafiles created or modified by a newer version of ArangoDB with an older ArangoDB version. For example, it is unsupported and is likely to cause problems when using 2.3 datafiles with an ArangoDB 2.2 instance.

Incompatible changes in ArangoDB 2.8

It is recommended to check the following list of incompatible changes before upgrading to ArangoDB 2.8, and adjust any client programs if necessary.

AQL

Keywords added

The following AQL keywords were added in ArangoDB 2.8:

- GRAPH

- OUTBOUND

- INBOUND

- ANY

- ALL

- NONE

- AGGREGATE

Usage of these keywords for collection names, variable names or attribute names in AQL queries will not be possible without quoting. For example, the following AQL query will still work as it uses a quoted collection name and a quoted attribute name:

FOR doc IN `OUTBOUND`

RETURN doc.`any`

Changed behavior

The AQL functions  NEAR  and  WITHIN  now have stricter validations for their input parameters  limit ,  radius  and  distance . They may now throw exceptions when invalid parameters are passed that may have not led to exceptions in previous versions.

Additionally, the expansion ( [*] ) operator in AQL has changed its behavior when handling non-array values:

In ArangoDB 2.8, calling the expansion operator on a non-array value will always return an empty array. Previous versions of ArangoDB expanded non-array values by calling the  TO_ARRAY()  function for the value, which for example returned an array with a single value for boolean, numeric and string input values, and an array with the object's values for an object input value. This behavior was inconsistent with how the expansion operator works for the array indexes in 2.8, so the behavior is now unified:

- if the left-hand side operand of  [*]  is an array, the array will be returned as is when calling  [*]  on it

- if the left-hand side operand of  [*]  is not an array, an empty array will be returned by  [*]

AQL queries that rely on the old behavior can be changed by either calling  TO_ARRAY  explicitly or by using the  [*]  at the correct position.

The following example query will change its result in 2.8 compared to 2.7:

LET values = "foo" RETURN values[*]

In 2.7 the query has returned the array  [ "foo" ] , but in 2.8 it will return an empty array  [ ] . To make it return the array  [ "foo" ]  again, an explicit  TO_ARRAY  function call is needed in 2.8 (which in this case allows the removal of the  [*]  operator altogether). This also works in 2.7:

LET values = "foo" RETURN TO_ARRAY(values)

Another example:

LET values = [ { name: "foo" }, { name: "bar" } ]

RETURN values[*].name[*]

The above returned  [ [ "foo" ], [ "bar" ] ] in 2.7. In 2.8 it will return [ [ ], [ ] ] , because the value of name` is not an array. To change the results to the 2.7 style, the query can be changed to

LET values = [ { name: "foo" }, { name: "bar" } ]

RETURN values[* RETURN TO_ARRAY(CURRENT.name)]

The above also works in 2.7. The following types of queries won't change:

LET values = [ 1, 2, 3 ] RETURN values[*]

LET values = [ { name: "foo" }, { name: "bar" } ] RETURN values[*].name

LET values = [ { names: [ "foo", "bar" ] }, { names: [ "baz" ] } ] RETURN values[*].names[*]

LET values = [ { names: [ "foo", "bar" ] }, { names: [ "baz" ] } ] RETURN values[*].names[**]

Deadlock handling

Client applications should be prepared to handle error 29 ( deadlock detected ) that ArangoDB may now throw when it detects a deadlock across multiple transactions. When a client application receives error 29, it should retry the operation that failed.

The error can only occur for AQL queries or user transactions that involve more than a single collection.

Optimizer

The AQL execution node type  IndexRangeNode  was replaced with a new more capable execution node type  IndexNode . That means in execution plan explain output there will be no more  IndexRangeNode s but only  IndexNode . This affects explain output that can be retrieved via  require("org/arangodb/aql/explainer").explain(query) ,  db._explain(query) , and the HTTP query explain API.

The optimizer rule that makes AQL queries actually use indexes was also renamed from  use-index-range  to  use-indexes . Again this affects explain output that can be retrieved via  require("org/arangodb/aql/explainer").explain(query) ,  db._explain(query) , and the HTTP query explain API.

The query optimizer rule  remove-collect-into  was renamed to  remove-collect-variables . This affects explain output that can be retrieved via  require("org/arangodb/aql/explainer").explain(query) ,  db._explain(query) , and the HTTP query explain API.

HTTP API

When a server-side operation got canceled due to an explicit client cancel request via HTTP  DELETE /_api/job , previous versions of ArangoDB returned an HTTP status code of 408 (request timeout) for the response of the canceled operation.

The HTTP return code 408 has caused problems with some client applications. Some browsers (e.g. Chrome) handled a 408 response by resending the original request, which is the opposite of what is desired when a job should be canceled.

Therefore ArangoDB will return HTTP status code 410 (gone) for canceled operations from version 2.8 on.

Foxx

Model and Repository

Due to compatibility issues the Model and Repository types are no longer implemented as ES2015 classes.

The pre-2.7 "extend" style subclassing is supported again and will not emit any deprecation warnings.

var Foxx = require('org/arangodb/foxx');

var MyModel = Foxx.Model.extend({

// ...

schema: {/* ... */}

});

Module resolution

The behavior of the JavaScript module resolution used by the  require  function has been modified to improve compatibility with modules written for Node.js.

Specifically

- absolute paths (e.g.  /some/absolute/path ) are now always interpreted as absolute file system paths, relative to the file system root

- global names (e.g.  global/name ) are now first intepreted as references to modules residing in a relevant  node_modules  folder, a built-in module or a matching document in the internal  _modules  collection, and only resolved to local file paths if no other match is found

Previously the two formats were treated interchangeably and would be resolved to local file paths first, leading to problems when local files used the same names as other modules (e.g. a local file  chai.js  would cause problems when trying to load the  chai  module installed in  node_modules ).

For more information see the blog announcement of this change and the upgrade guide.

Module  org/arangodb/request

The module now always returns response bodies, even for error responses. In versions prior to 2.8 the module would silently drop response bodies if the response header indicated an error.

The old behavior of not returning bodies for error responses can be restored by explicitly setting the option  returnBodyOnError  to  false :

let response = request({

//...

returnBodyOnError: false

});

Garbage collection

The V8 garbage collection strategy was slightly adjusted so that it eventually happens in all V8 contexts that hold V8 external objects (references to ArangoDB documents and collections). This enables a better cleanup of these resources and prevents other processes such as compaction being stalled while waiting for these resources to be released.

In this context the default value for the JavaScript garbage collection frequency ( --javascript.gc-frequency ) was also increased from 10 seconds to 15 seconds, as less internal operations in ArangoDB are carried out in JavaScript.

Client tools

arangodump will now fail by default when trying to dump edges that refer to already dropped collections. This can be circumvented by specifying the option  --force true  when invoking arangodump

Upgrading to ArangoDB 2.8

Please read the following sections if you upgrade from a previous version to ArangoDB 2.8. Please be sure that you have checked the list of changes in 2.8 before upgrading.

Please note first that a database directory used with ArangoDB 2.8 cannot be used with earlier versions (e.g. ArangoDB 2.7) any more. Upgrading a database directory cannot be reverted. Therefore please make sure to create a full backup of your existing ArangoDB installation before performing an upgrade.

Database Directory Version Check and Upgrade

ArangoDB will perform a database version check at startup. When ArangoDB 2.8 encounters a database created with earlier versions of ArangoDB, it will refuse to start. This is intentional.

The output will then look like this:

2015-12-04T17:11:17Z [31432] ERROR In database '_system': Database directory version (20702) is lower than current version (20800).

2015-12-04T17:11:17Z [31432] ERROR In database '_system': ----------------------------------------------------------------------

2015-12-04T17:11:17Z [31432] ERROR In database '_system': It seems like you have upgraded the ArangoDB binary.

2015-12-04T17:11:17Z [31432] ERROR In database '_system': If this is what you wanted to do, please restart with the

2015-12-04T17:11:17Z [31432] ERROR In database '_system':   --upgrade

2015-12-04T17:11:17Z [31432] ERROR In database '_system': option to upgrade the data in the database directory.

2015-12-04T17:11:17Z [31432] ERROR In database '_system': Normally you can use the control script to upgrade your database

2015-12-04T17:11:17Z [31432] ERROR In database '_system':   /etc/init.d/arangodb stop

2015-12-04T17:11:17Z [31432] ERROR In database '_system':   /etc/init.d/arangodb upgrade

2015-12-04T17:11:17Z [31432] ERROR In database '_system':   /etc/init.d/arangodb start

2015-12-04T17:11:17Z [31432] ERROR In database '_system': ----------------------------------------------------------------------

2015-12-04T17:11:17Z [31432] FATAL Database '_system' needs upgrade. Please start the server with the --upgrade option

To make ArangoDB 2.8 start with a database directory created with an earlier ArangoDB version, you may need to invoke the upgrade procedure once. This can be done by running ArangoDB from the command line and supplying the  --upgrade  option.

Note: here the same database should be specified that is also specified when arangod is started regularly. Please do not run the  --upgrade  command on each individual database subfolder (named  database-<some number> ).

For example, if you regularly start your ArangoDB server with

unix> arangod mydatabasefolder

then running

unix> arangod mydatabasefolder --upgrade

will perform the upgrade for the whole ArangoDB instance, including all of its databases.

Starting with  --upgrade  will run a database version check and perform any necessary migrations. As usual, you should create a backup of your database directory before performing the upgrade.

The last line of the output should look like this:

2015-12-04T17:12:15Z [31558] INFO database upgrade passed

Please check the full output the  --upgrade  run. Upgrading may produce errors, which need to be fixed before ArangoDB can be used properly. If no errors are present or they have been resolved manually, you can start ArangoDB 2.8 regularly.

Upgrading a cluster planned in the web interface

A cluster of ArangoDB instances has to be upgraded as well. This involves upgrading all ArangoDB instances in the cluster, as well as running the version check on the whole running cluster in the end.

We have tried to make this procedure as painless and convenient for you. We assume that you planned, launched and administrated a cluster using the graphical front end in your browser. The upgrade procedure is then as follows:

1. First shut down your cluster using the graphical front end as usual.

2. Then upgrade all dispatcher instances on all machines in your cluster using the version check as described above and restart them.

3. Now open the cluster dash board in your browser by pointing it to the same dispatcher that you used to plan and launch the cluster in the graphical front end. In addition to the usual buttons "Relaunch", "Edit cluster plan" and "Delete cluster plan" you will see another button marked "Upgrade and relaunch cluster".

4. Hit this button, your cluster will be upgraded and launched and all is done for you behind the scenes. If all goes well, you will see the usual cluster dash board after a few seconds. If there is an error, you have to inspect the log files of your cluster ArangoDB instances. Please let us know if you run into problems.

There is an alternative way using the  ArangoDB  shell. Instead of steps 3. and 4. above you can launch  arangosh , point it to the dispatcher that you have used to plan and launch the cluster using the option  --server.endpoint , and execute

arangosh> require("org/arangodb/cluster").Upgrade("root","");

This upgrades the cluster and launches it, exactly as with the button above in the graphical front end. You have to replace  "root"  with a user name and  ""  with a password that is valid for authentication with the cluster.

Upgrading Foxx apps generated by ArangoDB 2.7 and earlier

The implementation of the  require  function used to import modules in ArangoDB and Foxx has changed in order to improve compatibility with Node.js modules.

Given an app/service with the following layout:

- manifest.json

- controllers/

- todos.js

- models/

- todo.js

- repositories/

- todos.js

- node_modules/

- models/

- todo.js

The file  controllers/todos.js  would previously contain the following  require  calls:

var _ = require('underscore');

var joi = require('joi');

var Foxx = require('org/arangodb/foxx');

var ArangoError = require('org/arangodb').ArangoError;

var Todos = require('repositories/todos'); // <-- !

var Todo = require('models/todo'); // <-- !

The require paths  repositories/todos  and  models/todo  were previously resolved locally as relative to the app root.

Starting with 2.8 these paths would instead be resolved as relative to the  node_modules  folder or the global ArangoDB module paths before being resolved locally as a fallback.

In the given example layout the app would break in 2.8 because the module name  models/todo  would always resolve to  node_modules/models/todo.js  (which previously would have been ignored) instead of the local  models/todo.js .

In order to make sure the app still works in 2.8, the require calls in  controllers/todos.js  would need to be adjusted to look like this:

var _ = require('underscore');

var joi = require('joi');

var Foxx = require('org/arangodb/foxx');

var ArangoError = require('org/arangodb').ArangoError;

var Todos = require('../repositories/todos'); // <-- !

var Todo = require('../models/todo'); // <-- !

Note that the old "global" style require calls may still work in 2.8 but may break unexpectedly if modules with matching names are installed globally.

Incompatible changes in ArangoDB 2.7

It is recommended to check the following list of incompatible changes before upgrading to ArangoDB 2.7, and adjust any client programs if necessary.

AQL changes

DISTINCT  is now a keyword in AQL.

AQL queries that use  DISTINCT  (in lower, upper or mixed case) as an identifier (i.e. as a variable, a collection name or a function name) will stop working. To make such queries working again, each occurrence of  DISTINCT  in an AQL query should be enclosed in backticks. This will turn  DISTINCT  from a keyword into an identifier again.

The AQL function  SKIPLIST()  has been removed in ArangoDB 2.7. This function was deprecated in ArangoDB 2.6. It was a left-over from times when the query optimizer wasn't able to use skiplist indexes together with filters, skip and limit values. Since this issue been fixed since version 2.3, there is no AQL replacement function for  SKIPLIST . Queries that use the  SKIPLIST  function can be fixed by using the usual combination of  FOR ,  FILTER  and  LIMIT , e.g.

FOR doc IN @@collection

FILTER doc.value >= @value

SORT doc.value DESC

LIMIT 1

RETURN doc

Foxx changes

Bundling and compilation

The  assets  property is no longer supported in Foxx manifests and is scheduled to be removed in a future version of ArangoDB. The  files  property can still be used to serve static assets but it is recommended to use separate tooling to compile and bundle your assets.

Manifest scripts

The properties  setup  and  teardown  have been moved into the  scripts  property map:

Before:

{

...

"setup": "scripts/setup.js",

"teardown": "scripts/teardown.js"

}

After:

{

...

"scripts": {

"setup": "scripts/setup.js",

"teardown": "scripts/teardown.js"

}

}

Foxx Queues

Function-based Foxx Queue job types are no longer supported. To learn about how you can use the new script-based job types follow the updated recipe in the cookbook.

Foxx Sessions

The  jwt  and  type  options have been removed from the  activateSessions  API.

If you want to replicate the behavior of the  jwt  option you can use the JWT functions in the  crypto  module. A JWT-based session storage that doesn't write sessions to the database is available as the sessions-jwt app in the Foxx app store.

The session type is now inferred from the presence of the  cookie  or  header  options (allowing you to enable support for both). If you want to use the default settings for  cookie  or  header  you can pass the value  true  instead.

The  sessionStorageApp  option has been removed in favor of the  sessionStorage  option.

Before:

var Foxx = require('org/arangodb/foxx');

var ctrl = new Foxx.Controller(applicationContext);

ctrl.activateSessions({

sessionStorageApp: 'some-sessions-app',

type: 'cookie'

});

After:

ctrl.activateSessions({

sessionStorage: applicationContext.dependencies.sessions.sessionStorage,

cookie: true

});

Request module

The module  org/arangodb/request  uses an internal library function for sending HTTP requests. This library functionally unconditionally set an HTTP header  Accept-Encoding: gzip  in all outgoing HTTP requests, without client code having to set this header explicitly.

This has been fixed in 2.7, so  Accept-Encoding: gzip  is not set automatically anymore. Additionally the header  User-Agent: ArangoDB  is not set automatically either. If client applications rely on these headers being sent, they are free to add it when constructing requests using the request module.

The  internal.download()  function is also affected by this change. Again, the header can be added here if required by passing it via a  headers  sub-attribute in the third parameter ( options ) to this function.

arangodump / backups

The filenames in dumps created by arangodump now contain not only the name of the dumped collection, but also an additional 32-digit hash value. This is done to prevent overwriting dump files in case-insensitive file systems when there exist multiple collections with the same name (but with different cases).

This change leads to changed filenames in dumps created by arangodump. If any client scripts depend on the filenames in the dump output directory being equal to the collection name plus one of the suffixes  .structure.json  and  .data.json , they need to be adjusted.

Starting with ArangoDB 2.7, the file names will contain an underscore plus the 32-digit MD5 value (represented in hexadecimal notation) of the collection name.

For example, when arangodump dumps data of two collections test and Test, the filenames in previous versions of ArangoDB were:

- test.structure.json  (definitions for collection test)

- test.data.json  (data for collection test)

- Test.structure.json  (definitions for collection Test)

- Test.data.json  (data for collection Test)

In 2.7, the filenames will be:

- test_098f6bcd4621d373cade4e832627b4f6.structure.json  (definitions for collection test)

- test_098f6bcd4621d373cade4e832627b4f6.data.json  (data for collection test)

- Test_0cbc6611f5540bd0809a388dc95a615b.structure.json  (definitions for collection Test)

- Test_0cbc6611f5540bd0809a388dc95a615b.data.json  (data for collection Test)

Starting / stopping

When starting arangod, the server will now drop the process privileges to the specified values in options  --server.uid  and  --server.gid  instantly after parsing the startup options.

That means when either  --server.uid  or  --server.gid  are set, the privilege change will happen earlier. This may prevent binding the server to an endpoint with a port number lower than 1024 if the arangodb user has no privileges for that. Previous versions of ArangoDB changed the privileges later, so some startup actions were still carried out under the invoking user (i.e. likely root when started via init.d or system scripts) and especially binding to low port numbers was still possible there.

The default privileges for user arangodb will not be sufficient for binding to port numbers lower than 1024. To have an ArangoDB 2.7 bind to a port number lower than 1024, it needs to be started with either a different privileged user, or the privileges of the arangodb user have to raised manually beforehand.

Additionally, Linux startup scripts and systemd configuration for arangod now will adjust the NOFILE (number of open files) limits for the process. The limit value is set to 131072 (128k) when ArangoDB is started via start/stop commands. The goal of this change is to prevent arangod from running out of available file descriptors for socket connections and datafiles.

Connection handling

arangod will now actually close lingering client connections when idle for at least the duration specified in the  --server.keep-alive-timeout  startup option.

In previous versions of ArangoDB, idle connections were not closed by the server when the timeout was reached and the client was still connected. Now the connection is properly closed by the server in case of timeout. Client applications relying on the old behavior may now need to reconnect to the server when their idle connections time out and get closed (note: connections being idle for a long time may be closed by the OS or firewalls anyway - client applications should be aware of that and try to reconnect).

Option changes

Configure options removed

The following options for  configure  have been removed because they were unused or exotic:

- --enable-timings

- --enable-figures

Startup options added

The following configuration options have been added in 2.7:

- --database.query-cache-max-results : sets the maximum number of results in AQL query result cache per database

- --database.query-cache-mode : sets the mode for the AQL query results cache. Possible values are  on ,  off  and  demand . The default value is  off

Miscellaneous changes

Simple queries

Many simple queries provide a  skip()  function that can be used to skip over a certain number of documents in the result. This function allowed specifying negative offsets in previous versions of ArangoDB. Specifying a negative offset led to the query result being iterated in reverse order, so skipping was performed from the back of the result. As most simple queries do not provide a guaranteed result order, skipping from the back of a result with unspecific order seems a rather exotic use case and was removed to increase consistency with AQL, which also does not provide negative skip values.

Negative skip values were deprecated in ArangoDB 2.6.

Tasks API

The undocumented function  addJob()  has been removed from the  org/arangodb/tasks  module in ArangoDB 2.7.

Runtime endpoints manipulation API

The following HTTP REST API methods for runtime manipulation of server endpoints have been removed in ArangoDB 2.7:

- POST  /_api/endpoint : to dynamically add an endpoint while the server was running

- DELETE  /_api/endpoint : to dynamically remove an endpoint while the server was running

This change also affects the equivalent JavaScript endpoint manipulation methods available in Foxx. The following functions have been removed in ArangoDB 2.7:

- db._configureEndpoint()

- db._removeEndpoint()

Incompatible changes in ArangoDB 2.6

It is recommended to check the following list of incompatible changes before upgrading to ArangoDB 2.6, and adjust any client programs if necessary.

Requirements

ArangoDB's built-in web interface now uses cookies for session management. Session information ids are stored in cookies, so clients using the web interface must accept cookies in order to log in and use it.

Foxx changes

Foxx Queues

Foxx Queue job type definitions were previously based on functions and had to be registered before use. Due to changes in 2.5 this resulted in problems when restarting the server or defining job types incorrectly.

Function-based job types have been deprecated in 2.6 and will be removed entirely in 2.7.

In order to convert existing function-based job types to the new script-based job types, create custom scripts in your Foxx app and reference them by their name and the mount point of the app they are defined in. Official job types from the Foxx app store can be upgraded by upgrading from the 1.x version to the 2.x version of the same app.

In order to upgrade queued jobs to the new job types, you need to update the  type  property of the affected jobs in the database's  _jobs  system collection. In order to see the collection in the web interface you need to enable the collection type "System" in the collection list options.

Example:

Before:  "type": "mailer.postmark"

After:  "type": {"name": "mailer", "mount": "/my-postmark-mailer"}

Foxx Sessions

The options  jwt  and  type  of the controller method  controller.activateSessions  have been deprecated in 2.6 and will be removed entirely in 2.7.

If you want to use pure JWT sessions, you can use the  sessions-jwt  Foxx app from the Foxx app store.

If you want to use your own JWT-based sessions, you can use the JWT functions in the  crypto  module directly.

Instead of using the  type  option you can just use the  cookie  and  header  options on their own, which both now accept the value  true  to enable them with their default configurations.

The option  sessionStorageApp  has been renamed to  sessionStorage  and now also accepts session storages directly. The old option  sessionStorageApp  will be removed entirely in 2.7.

Libraries

The bundled version of the  joi  library used in Foxx was upgraded to version 6.0.8. This may affect Foxx applications that depend on the library.

AQL changes

AQL LENGTH function

The return value of the AQL  LENGTH  function was changed if  LENGTH  is applied on  null  or a boolean value:

- LENGTH(null)  now returns  0 . In previous versions of ArangoDB, this returned  4 .

- LENGTH(false)  now returns  0 . In previous versions of ArangoDB, the return value was  5 .

- LENGTH(true)  now returns  1 . In previous versions of ArangoDB, the return value was  4 .

AQL graph functions

In 2.6 the graph functions did undergo a performance lifting. During this process we had to adopt the result format and the options for some of them. Many graph functions now have an option  includeData  which allows to trigger if the result of this function should contain fully extracted documents  includeData: true  or only the  _id  values  includeData: false . In most use cases the  _id  is sufficient to continue and the extraction of data is an unnecessary operation. The AQL functions supporting this additional option are:

- SHORTEST_PATH

- NEIGHBORS

- GRAPH_SHORTEST_PATH

- GRAPH_NEIGHBORS

- GRAPH_EDGES

Furthermore the result  SHORTEST_PATH  has changed. The old format returned a list of all vertices on the path. Optionally it could include each sub-path for these vertices. All of the documents were fully extracted. Example:

[

{

vertex: {

_id: "vertex/1",

_key: "1",

_rev: "1234"

name: "Alice"

},

path: {

vertices: [

{

_id: "vertex/1",

_key: "1",

_rev: "1234"

name: "Alice"

}

],

edges: []

}

},

{

vertex: {

_id: "vertex/2",

_key: "2",

_rev: "5678"

name: "Bob"

},

path: {

vertices: [

{

_id: "vertex/1",

_key: "1",

_rev: "1234"

name: "Alice"

}, {

_id: "vertex/2",

_key: "2",

_rev: "5678"

name: "Bob"

}

],

edges: [

{

_id: "edge/1",

_key: "1",

_rev: "9876",

type: "loves"

}

]

}

}

]

The new version is more compact. Each  SHORTEST_PATH  will only return one document having the attributes  vertices ,  edges ,  distance . The  distance  is computed taking into account the given weight. Optionally the documents can be extracted with  includeData: true  Example:

{

vertices: [

"vertex/1",

"vertex/2"

],

edges: [

"edge/1"

],

distance: 1

}

The next function that returns a different format is  NEIGHBORS . Since 2.5 it returned an object with  edge  and  vertex  for each connected edge. Example:

[

{

vertex: {

_id: "vertex/2",

_key: "2",

_rev: "5678"

name: "Bob"

},

edge: {

_id: "edge/1",

_key: "1",

_rev: "9876",

type: "loves"

}

}

]

With 2.6 it will only return the vertex directly, again using  includeData: true . By default it will return a distinct set of neighbors, using the option  distinct: false  will include the same vertex for each edge pointing to it.

Example:

[

"vertex/2"

]

Function and API changes

Graph measurements functions

All graph measurements functions in JavaScript module  general-graph  that calculated a single figure previously returned an array containing just the figure. Now these functions will return the figure directly and not put it inside an array.

The affected functions are:

- graph._absoluteEccentricity

- graph._eccentricity

- graph._absoluteCloseness

- graph._closeness

- graph._absoluteBetweenness

- graph._betweenness

- graph._radius

- graph._diameter

Client programs calling these functions should be adjusted so they process the scalar value returned by the function instead of the previous array value.

Cursor API

A batchSize value  0  is now disallowed when calling the cursor API via HTTP  POST /_api/cursor .

The HTTP REST API  POST /_api/cursor  does not accept a  batchSize  parameter value of  0  any longer. A batch size of 0 never made much sense, but previous versions of ArangoDB did not check for this value. Now creating a cursor using a  batchSize  value 0 will result in an HTTP 400 error response.

Document URLs returned

The REST API method GET  /_api/document?collection=...  (that method will return partial URLs to all documents in the collection) will now properly prefix document address URLs with the current database name.

Previous versions of ArangoDB returned the URLs starting with  /_api/  but without the current database name, e.g.  /_api/document/mycollection/mykey . Starting with 2.6, the response URLs will include the database name as well, e.g.  /_db/_system/_api/document/mycollection/mykey .

Fulltext indexing

Fulltext indexes will now also index text values contained in direct sub-objects of the indexed attribute.

Previous versions of ArangoDB only indexed the attribute value if it was a string. Sub-attributes of the index attribute were ignored when fulltext indexing.

Now, if the index attribute value is an object, the object's values will each be included in the fulltext index if they are strings. If the index attribute value is an array, the array's values will each be included in the fulltext index if they are strings.

Deprecated server functionality

Simple queries

The following simple query functions are now deprecated:

- collection.near

- collection.within

- collection.geo

- collection.fulltext

- collection.range

- collection.closedRange

This also lead to the following REST API methods being deprecated from now on:

- PUT /_api/simple/near

- PUT /_api/simple/within

- PUT /_api/simple/fulltext

- PUT /_api/simple/range

It is recommended to replace calls to these functions or APIs with equivalent AQL queries, which are more flexible because they can be combined with other operations:

FOR doc IN NEAR(@@collection, @latitude, @longitude, @limit)

RETURN doc

FOR doc IN WITHIN(@@collection, @latitude, @longitude, @radius, @distanceAttributeName)

RETURN doc

FOR doc IN FULLTEXT(@@collection, @attributeName, @queryString, @limit)

RETURN doc

FOR doc IN @@collection

FILTER doc.value >= @left && doc.value < @right

LIMIT @skip, @limit

RETURN doc`

The above simple query functions and REST API methods may be removed in future versions of ArangoDB.

Using negative values for  SimpleQuery.skip()  is also deprecated. This functionality will be removed in future versions of ArangoDB.

AQL functions

The AQL  SKIPLIST  function has been deprecated because it is obsolete.

The function was introduced in older versions of ArangoDB with a less powerful query optimizer to retrieve data from a skiplist index using a  LIMIT  clause.

Since 2.3 the same goal can be achieved by using regular AQL constructs, e.g.

FOR doc IN @@collection

FILTER doc.value >= @value

SORT doc.value

LIMIT 1

RETURN doc

Startup option changes

Options added

The following configuration options have been added in 2.6:

- --server.session-timeout : allows controlling the timeout of user sessions in the web interface. The value is specified in seconds.

- --server.foxx-queues : controls whether the Foxx queue manager will check queue and job entries. Disabling this option can reduce server load but will prevent jobs added to Foxx queues from being processed at all.

The default value is  true , enabling the Foxx queues feature.

- --server.foxx-queues-poll-interval : allows adjusting the frequency with which the Foxx queues manager is checking the queue (or queues) for jobs to be executed.

The default value is  1  second. Lowering this value will result in the queue manager waking up and checking the queues more frequently, which may increase CPU usage of the server.

Note: this option only has an effect when  --server.foxx-queues  is not set to  false .

Options removed

The following configuration options have been removed in 2.6.:

- --log.severity : the docs for  --log.severity  mentioned lots of severities (e.g.  exception ,  technical ,  functional ,  development ) but only a few severities (e.g.  all ,  human ) were actually used, with  human  being the default and  all  enabling the additional logging of incoming requests.

The option pretended to control a lot of things which it actually didn't. Additionally, the option  --log.requests-file  was around for a long time already, also controlling request logging.

Because the  --log.severity  option effectively did not control that much, it was removed. A side effect of removing the option is that 2.5 installations started with option  --log.severity all  will not log requests after the upgrade to 2.6. This can be adjusted by setting the  --log.requests-file  option instead.

Default values changed

The default values for the following options have changed in 2.6:

- --database.ignore-datafile-errors : the default value for this option was changed from  true  to  false .

If the new default value of  false  is used, then arangod will refuse loading collections that contain datafiles with CRC mismatches or other errors. A collection with datafile errors will then become unavailable. This prevents follow up errors from happening.

The only way to access such collection is to use the datafile debugger (arango-dfdb) and try to repair or truncate the datafile with it.

- --server.request-timeout : the default value was increased from 300 to 1200 seconds for all client tools (arangosh, arangoimp, arangodump, arangorestore).

- --server.connect-timeout : the default value was increased from 3 to 5 seconds for all client tools (arangosh, arangoimp, arangodump, arangorestore).

Upgrading to ArangoDB 2.6

Please read the following sections if you upgrade from a previous version to ArangoDB 2.6. Please be sure that you have checked the list of changes in 2.6 before upgrading.

Please note first that a database directory used with ArangoDB 2.6 cannot be used with earlier versions (e.g. ArangoDB 2.5) any more. Upgrading a database directory cannot be reverted. Therefore please make sure to create a full backup of your existing ArangoDB installation before performing an upgrade.

Database Directory Version Check and Upgrade

ArangoDB will perform a database version check at startup. When ArangoDB 2.6 encounters a database created with earlier versions of ArangoDB, it will refuse to start. This is intentional.

The output will then look like this:

2015-02-17T09:43:11Z [8302] ERROR In database '_system': Database directory version (20501) is lower than current version (20600).

2015-02-17T09:43:11Z [8302] ERROR In database '_system': ----------------------------------------------------------------------

2015-02-17T09:43:11Z [8302] ERROR In database '_system': It seems like you have upgraded the ArangoDB binary.

2015-02-17T09:43:11Z [8302] ERROR In database '_system': If this is what you wanted to do, please restart with the

2015-02-17T09:43:11Z [8302] ERROR In database '_system':   --upgrade

2015-02-17T09:43:11Z [8302] ERROR In database '_system': option to upgrade the data in the database directory.

2015-02-17T09:43:11Z [8302] ERROR In database '_system': Normally you can use the control script to upgrade your database

2015-02-17T09:43:11Z [8302] ERROR In database '_system':   /etc/init.d/arangodb stop

2015-02-17T09:43:11Z [8302] ERROR In database '_system':   /etc/init.d/arangodb upgrade

2015-02-17T09:43:11Z [8302] ERROR In database '_system':   /etc/init.d/arangodb start

2015-02-17T09:43:11Z [8302] ERROR In database '_system': ----------------------------------------------------------------------

2015-02-17T09:43:11Z [8302] FATAL Database '_system' needs upgrade. Please start the server with the --upgrade option

To make ArangoDB 2.6 start with a database directory created with an earlier ArangoDB version, you may need to invoke the upgrade procedure once. This can be done by running ArangoDB from the command line and supplying the  --upgrade  option.

Note: here the same database should be specified that is also specified when arangod is started regularly. Please do not run the  --upgrade  command on each individual database subfolder (named  database-<some number> ).

For example, if you regularly start your ArangoDB server with

unix> arangod mydatabasefolder

then running

unix> arangod mydatabasefolder --upgrade

will perform the upgrade for the whole ArangoDB instance, including all of its databases.

Starting with  --upgrade  will run a database version check and perform any necessary migrations. As usual, you should create a backup of your database directory before performing the upgrade.

The last line of the output should look like this:

2014-12-22T12:03:31Z [12026] INFO database upgrade passed

Please check the full output the  --upgrade  run. Upgrading may produce errors, which need to be fixed before ArangoDB can be used properly. If no errors are present or they have been resolved manually, you can start ArangoDB 2.6 regularly.

Upgrading a cluster planned in the web interface

A cluster of ArangoDB instances has to be upgraded as well. This involves upgrading all ArangoDB instances in the cluster, as well as running the version check on the whole running cluster in the end.

We have tried to make this procedure as painless and convenient for you. We assume that you planned, launched and administrated a cluster using the graphical front end in your browser. The upgrade procedure is then as follows:

1. First shut down your cluster using the graphical front end as usual.

2. Then upgrade all dispatcher instances on all machines in your cluster using the version check as described above and restart them.

3. Now open the cluster dash board in your browser by pointing it to the same dispatcher that you used to plan and launch the cluster in the graphical front end. In addition to the usual buttons "Relaunch", "Edit cluster plan" and "Delete cluster plan" you will see another button marked "Upgrade and relaunch cluster".

4. Hit this button, your cluster will be upgraded and launched and all is done for you behind the scenes. If all goes well, you will see the usual cluster dash board after a few seconds. If there is an error, you have to inspect the log files of your cluster ArangoDB instances. Please let us know if you run into problems.

There is an alternative way using the  ArangoDB  shell. Instead of steps 3. and 4. above you can launch  arangosh , point it to the dispatcher that you have used to plan and launch the cluster using the option  --server.endpoint , and execute

arangosh> require("org/arangodb/cluster").Upgrade("root","");

This upgrades the cluster and launches it, exactly as with the button above in the graphical front end. You have to replace  "root"  with a user name and  ""  with a password that is valid for authentication with the cluster.

Incompatible changes in ArangoDB 2.5

It is recommended to check the following list of incompatible changes before upgrading to ArangoDB 2.5, and adjust any client programs if necessary.

Changed behavior

V8

The V8 version shipped with ArangoDB was upgraded from 3.29.59 to 3.31.74.1. This leads to additional ECMAScript 6 (ES6 or "harmony") features being enabled by default in ArangoDB's scripting environment.

Apart from that, a change in the interpretation of command-line options by V8 may affect users. ArangoDB passes the value of the command-line option  --javascript.v8-options  to V8 and leaves interpretation of the contents to V8. For example, the ArangoDB option  --javascript.v8-options="--harmony"  could be used to tell V8 to enable its harmony features.

In ArangoDB 2.4, the following harmony options were made available by V8:

- --harmony_scoping (enable harmony block scoping)

- --harmony_modules (enable harmony modules (implies block scoping))

- --harmony_proxies (enable harmony proxies)

- --harmony_generators (enable harmony generators)

- --harmony_numeric_literals (enable harmony numeric literals (0o77, 0b11))

- --harmony_strings (enable harmony string)

- --harmony_arrays (enable harmony arrays)

- --harmony_arrow_functions (enable harmony arrow functions)

- --harmony_classes (enable harmony classes)

- --harmony_object_literals (enable harmony object literal extensions)

- --harmony (enable all harmony features (except proxies))

There was the option  --harmony , which turned on almost all harmony features.

In ArangoDB 2.5, V8 provides the following harmony-related options:

- --harmony (enable all completed harmony features)

- --harmony_shipping (enable all shipped harmony features)

- --harmony_modules (enable "harmony modules (implies block scoping)" (in progress))

- --harmony_arrays (enable "harmony array methods" (in progress))

- --harmony_array_includes (enable "harmony Array.prototype.includes" (in progress))

- --harmony_regexps (enable "harmony regular expression extensions" (in progress))

- --harmony_arrow_functions (enable "harmony arrow functions" (in progress))

- --harmony_proxies (enable "harmony proxies" (in progress))

- --harmony_sloppy (enable "harmony features in sloppy mode" (in progress))

- --harmony_unicode (enable "harmony unicode escapes" (in progress))

- --harmony_tostring (enable "harmony toString")

- --harmony_numeric_literals (enable "harmony numeric literals")

- --harmony_strings (enable "harmony string methods")

- --harmony_scoping (enable "harmony block scoping")

- --harmony_classes (enable "harmony classes (implies block scoping & object literal extension)")

- --harmony_object_literals (enable "harmony object literal extensions")

- --harmony_templates (enable "harmony template literals")

Note that there are extra options for better controlling the dedicated features, and especially that the meaning of the  --harmony  option has changed from enabling all harmony features to all completed harmony features!

Users should adjust the value of  --javascript.v8-options  accordingly.

Please note that incomplete harmony features are subject to change in future V8 releases.

Sparse indexes

Hash indexes and skiplist indexes can now be created in a sparse variant. When not explicitly set, the  sparse  attribute defaults to  false  for new indexes.

This causes a change in behavior when creating a unique hash index without specifying the sparse flag. The unique hash index will be created in a non-sparse variant in ArangoDB 2.5.

In 2.4 and before, unique hash indexes were implicitly sparse, always excluding  null  values from the index. There was no option to control this behavior, and sparsity was neither supported for non-unique hash indexes nor skiplists in 2.4. This implicit sparsity of just unique hash indexes was considered an inconsistency, and therefore the behavior was cleaned up in 2.5.

As of 2.5, hash and skiplist indexes will only be created sparse if sparsity is explicitly requested. This may require a change in index-creating client code, but only if the client code creates unique hash indexes and if they are still intended to be sparse. In this case, the client code should explicitly set the  sparse  flag to  true  when creating a unique hash index.

Existing unique hash indexes from 2.4 or before will automatically be migrated so they are still sparse after the upgrade to 2.5. For these indexes, the  sparse  attribute will be populated automatically with a value of  true .

Geo indexes are implicitly sparse, meaning documents without the indexed location attribute or containing invalid location coordinate values will be excluded from the index automatically. This is also a change when compared to pre-2.5 behavior, when documents with missing or invalid coordinate values may have caused errors on insertion when the geo index'  unique  flag was set and its  ignoreNull  flag was not.

This was confusing and has been rectified in 2.5. The method  ensureGeoConstaint()  now does the same as  ensureGeoIndex() . Furthermore, the attributes  constraint ,  unique ,  ignoreNull  and  sparse  flags are now completely ignored when creating geo indexes. Client index creation code therefore does not need to set the  ignoreNull  or  constraint  attributes when creating a geo index.

The same is true for fulltext indexes. There is no need to specify non-uniqueness or sparsity for geo or fulltext indexes. They will always be non-unique and sparse.

Moved Foxx applications to a different folder.

Until 2.4 foxx apps were stored in the following folder structure:  <app-path>/databases/<dbname>/<appname>:<appversion> . This caused some trouble as apps where cached based on name and version and updates did not apply. Also the path on filesystem and the app's access URL had no relation to one another. Now the path on filesystem is identical to the URL (except the appended APP):  <app-path>/_db/<dbname>/<mointpoint>/APP

Foxx Development mode

The development mode used until 2.4 is gone. It has been replaced by a much more mature version. This includes the deprecation of the javascript.dev-app-path parameter, which is useless since 2.5. Instead of having two separate app directories for production and development, apps now reside in one place, which is used for production as well as for development. Apps can still be put into development mode, changing their behavior compared to production mode. Development mode apps are still reread from disk at every request, and still they ship more debug output.

This change has also made the startup options  --javascript.frontend-development-mode  and  --javascript.dev-app-path  obsolete. The former option will not have any effect when set, and the latter option is only read and used during the upgrade to 2.5 and does not have any effects later.

Foxx install process

Installing Foxx apps has been a two step process: import them into ArangoDB and mount them at a specific mountpoint. These operations have been joined together. You can install an app at one mountpoint, that's it. No fetch, mount, unmount, purge cycle anymore. The commands have been simplified to just:

- install: get your Foxx app up and running

- uninstall: shut it down and erase it from disk

Deprecated features

- Foxx: method  Model#toJSONSchema(id)  is deprecated, it will raise a warning if you use it. Please use  Foxx.toJSONSchema(id, model)  instead.

Removed features

- Startup switch  --javascript.frontend-development-mode : Its major purpose was internal development anyway. Now the web frontend can be set to development mode similar to any other foxx app.

- Startup switch  --javascript.dev-app-path : Was used for the development mode of Foxx. This is integrated with the normal app-path now and can be triggered on app level. The second app-path is superfluous.

- Foxx:  controller.collection : Please use  appContext.collection  instead.

- Foxx:  FoxxRepository.modelPrototype : Please use  FoxxRepository.model  instead.

- Foxx:  Model.extend({}, {attributes: {}}) : Please use  Model.extend({schema: {}})  instead.

- Foxx:  requestContext.bodyParam(paramName, description, Model) : Please use  requestContext.bodyParam(paramName, options)  instead.

- Foxx:  requestContext.queryParam({type: string}) : Please use  requestContext.queryParam({type: joi})  instead.

- Foxx:  requestContext.pathParam({type: string}) : Please use  requestContext.pathParam({type: joi})  instead.

- Graph: The modules  org/arangodb/graph  and  org/arangodb/graph-blueprint : Please use module  org/arangodb/general-graph  instead. NOTE: This does not mean we do not support blueprints any more. General graph covers everything the graph--blueprint did, plus many more features.

- General-Graph: In the module  org/arangodb/general-graph  the functions  _undirectedRelation  and  _directedRelation  are no longer available. Both functions have been unified to  _relation .

Upgrading to ArangoDB 2.5

Please read the following sections if you upgrade from a previous version to ArangoDB 2.5. Please be sure that you have checked the list of changes in 2.5 before upgrading.

Please note first that a database directory used with ArangoDB 2.5 cannot be used with earlier versions (e.g. ArangoDB 2.4) any more. Upgrading a database directory cannot be reverted. Therefore please make sure to create a full backup of your existing ArangoDB installation before performing an upgrade.

In 2.5 we have also changed the paths for Foxx applications. Please also make sure that you have a backup of all Foxx apps in your  javascript.app-path  and  javascript.dev-app-path . It is sufficient to have the source files for Foxx somewhere else so you can reinstall them on error. To check that everything has worked during upgrade you could use the web-interface Applications tab or

unix> foxx-manager list

for all your databases. The listed apps should be identical before and after the upgrade.

Database Directory Version Check and Upgrade

ArangoDB will perform a database version check at startup. When ArangoDB 2.5 encounters a database created with earlier versions of ArangoDB, it will refuse to start. This is intentional.

The output will then look like this:

2015-02-17T09:43:11Z [8302] ERROR In database '_system': Database directory version (20401) is lower than current version (20500).

2015-02-17T09:43:11Z [8302] ERROR In database '_system': ----------------------------------------------------------------------

2015-02-17T09:43:11Z [8302] ERROR In database '_system': It seems like you have upgraded the ArangoDB binary.

2015-02-17T09:43:11Z [8302] ERROR In database '_system': If this is what you wanted to do, please restart with the

2015-02-17T09:43:11Z [8302] ERROR In database '_system':   --upgrade

2015-02-17T09:43:11Z [8302] ERROR In database '_system': option to upgrade the data in the database directory.

2015-02-17T09:43:11Z [8302] ERROR In database '_system': Normally you can use the control script to upgrade your database

2015-02-17T09:43:11Z [8302] ERROR In database '_system':   /etc/init.d/arangodb stop

2015-02-17T09:43:11Z [8302] ERROR In database '_system':   /etc/init.d/arangodb upgrade

2015-02-17T09:43:11Z [8302] ERROR In database '_system':   /etc/init.d/arangodb start

2015-02-17T09:43:11Z [8302] ERROR In database '_system': ----------------------------------------------------------------------

2015-02-17T09:43:11Z [8302] FATAL Database '_system' needs upgrade. Please start the server with the --upgrade option

To make ArangoDB 2.5 start with a database directory created with an earlier ArangoDB version, you may need to invoke the upgrade procedure once. This can be done by running ArangoDB from the command line and supplying the  --upgrade  option. Note: We have changed Foxx folder structure and implemented an upgrade task to move your applications to the new structure. In order to tell this upgrade task to also move your development Foxx apps please make sure you give the dev-app-path as well. If you have not used development mode for Foxx apps you can drop the  --javascript.dev-app-path . It is only possible to upgrade one dev-app-path together with one data folder.

unix> arangod data --upgrade --javascript.dev-app-path devapps

where  data  is ArangoDB's main data directory and  devapps  is the directory where you develop Foxx apps.

Note: here the same database should be specified that is also specified when arangod is started regularly. Please do not run the  --upgrade  command on each individual database subfolder (named  database-<some number> ).

For example, if you regularly start your ArangoDB server with

unix> arangod mydatabasefolder

then running

unix> arangod mydatabasefolder --upgrade

will perform the upgrade for the whole ArangoDB instance, including all of its databases.

Starting with  --upgrade  will run a database version check and perform any necessary migrations. As usual, you should create a backup of your database directory before performing the upgrade.

The last line of the output should look like this:

2014-12-22T12:03:31Z [12026] INFO database upgrade passed

Please check the full output the  --upgrade  run. Upgrading may produce errors, which need to be fixed before ArangoDB can be used properly. If no errors are present or they have been resolved manually, you can start ArangoDB 2.5 regularly.

Upgrading a cluster planned in the web interface

A cluster of ArangoDB instances has to be upgraded as well. This involves upgrading all ArangoDB instances in the cluster, as well as running the version check on the whole running cluster in the end.

We have tried to make this procedure as painless and convenient for you. We assume that you planned, launched and administrated a cluster using the graphical front end in your browser. The upgrade procedure is then as follows:

1. First shut down your cluster using the graphical front end as usual.

2. Then upgrade all dispatcher instances on all machines in your cluster using the version check as described above and restart them.

3. Now open the cluster dash board in your browser by pointing it to the same dispatcher that you used to plan and launch the cluster in the graphical front end. In addition to the usual buttons "Relaunch", "Edit cluster plan" and "Delete cluster plan" you will see another button marked "Upgrade and relaunch cluster".

4. Hit this button, your cluster will be upgraded and launched and all is done for you behind the scenes. If all goes well, you will see the usual cluster dash board after a few seconds. If there is an error, you have to inspect the log files of your cluster ArangoDB instances. Please let us know if you run into problems.

There is an alternative way using the  ArangoDB  shell. Instead of steps 3. and 4. above you can launch  arangosh , point it to the dispatcher that you have used to plan and launch the cluster using the option  --server.endpoint , and execute

arangosh> require("org/arangodb/cluster").Upgrade("root","");

This upgrades the cluster and launches it, exactly as with the button above in the graphical front end. You have to replace  "root"  with a user name and  ""  with a password that is valid for authentication with the cluster.

Incompatible changes in ArangoDB 2.4

It is recommended to check the following list of incompatible changes before upgrading to ArangoDB 2.4, and adjust any client programs if necessary.

Changed behavior

V8 upgrade

The bundled V8 version has been upgraded from 3.16.14 to 3.29.59.

The new version provides better error checking, which can lead to subtle changes in the execution of JavaScript code.

The following code, though nonsense, runs without error in 2.3 and 2.4 when strict mode is not enabled:

(function () {

a = true;

a.foo = 1;

})();

When enabling strict mode, the function will throw an error in 2.4 but not in 2.3:

(function () {

"use strict";

a = true;

a.foo = 1;

})();

TypeError: Cannot assign to read only property 'foo' of true

Though this is a change in behavior it can be considered an improvement. The new version actually uncovers an error that went undetected in the old version.

Error messages have also changed slightly in the new version. Applications that rely on the exact error messages of the JavaScript engine may need to be adjusted so they look for the updated error messages.

Default endpoint

The default endpoint for arangod is now  127.0.0.1 .

This change will modify the IP address ArangoDB listens on to 127.0.0.1 by default. This will make new ArangoDB installations unaccessible from clients other than localhost unless the configuration is changed. This is a security feature.

To make ArangoDB accessible from any client, change the server's configuration ( --server.endpoint ) to either  tcp://0.0.0.0:8529  or the server's publicly visible IP address.

Replication

System collections are now included in the replication and all replication API return values by default.

This will lead to user accounts and credentials data being replicated from master to slave servers. This may overwrite slave-specific database users.

This may be considered a feature or an anti-feature, so it is configurable.

If replication of system collections is undesired, they can be excluded from replication by setting the  includeSystem  attribute to  false  in the following commands:

- initial synchronization:  replication.sync({ includeSystem: false })

- continuous replication:  replication.applier.properties({ includeSystem: false })

This will exclude all system collections (including  _aqlfunctions ,  _graphs  etc.) from the initial synchronization and the continuous replication.

If this is also undesired, it is also possible to specify a list of collections to exclude from the initial synchronization and the continuous replication using the  restrictCollections  attribute, e.g.:

require("org/arangodb/replication").applier.properties({

includeSystem: true,

restrictType: "exclude",

restrictCollections: [ "_users", "_graphs", "foo" ]

});

The above example will in general include system collections, but will exclude the specified three collections from continuous replication.

The HTTP REST API methods for fetching the replication inventory and for dumping collections also support the  includeSystem  control flag via a URL parameter of the same name.

Build process changes

Several options for the  configure  command have been removed in 2.4. The options

- --enable-all-in-one-v8

- --enable-all-in-one-icu

- --enable-all-in-one-libev

- --with-libev=DIR

- --with-libev-lib=DIR

- --with-v8=DIR

- --with-v8-lib=DIR

- --with-icu-config=FILE

are not available anymore because the build process will always use the bundled versions of the libraries.

When building ArangoDB from source in a directory that already contained a pre-2.4 version, it will be necessary to run a  make superclean  command once and a full rebuild afterwards:

git pull

make superclean

make setup

./configure <options go here>

make

Miscellaneous changes

As a consequence of global renaming in the codebase, the option  mergeArrays  has been renamed to  mergeObjects . This option controls whether JSON objects will be merged on an update operation or overwritten. The default has been, and still is, to merge. Not specifying the parameter will lead to a merge, as it has been the behavior in ArangoDB ever since.

This affects the HTTP REST API method PATCH  /_api/document/collection/key . Its optional URL parameter  mergeArrays  for the option has been renamed to  mergeObjects .

The AQL  UPDATE  statement is also affected, as its option  mergeArrays  has also been renamed to  mergeObjects . The 2.3 query

UPDATE doc IN collection WITH { ... } IN collection OPTIONS { mergeArrays: false }

should thus be rewritten to the following in 2.4:

UPDATE doc IN collection WITH { ... } IN collection OPTIONS { mergeObjects: false }

Deprecated features

For  FoxxController  objects, the method  collection()  is deprecated and will be removed in future version of ArangoDB. Using this method will issue a warning. Please use  applicationContext.collection()  instead.

For  FoxxRepository  objects, the property  modelPrototype  is now deprecated. Using it will issue a warning. Please use  FoxxRepository.model  instead.

In  FoxxController  /  RequestContext , calling method  bodyParam()  with three arguments is deprecated. Please use  .bodyParam(paramName, options)  instead.

In  FoxxController  /  RequestContext  calling method  queryParam({type: string})  is deprecated. Please use  requestContext.queryParam({type: joi})  instead.

In  FoxxController  /  RequestContext  calling method  pathParam({type: string})  is deprecated. Please use  requestContext.pathParam({type: joi})  instead.

For  FoxxModel , calling  Model.extend({}, {attributes: {}})  is deprecated. Please use  Model.extend({schema: {}})  instead.

In module  org/arangodb/general-graph , the functions  _undirectedRelation()  and  _directedRelation()  are deprecated and will be removed in a future version of ArangoDB. Both functions have been unified to  _relation() .

The modules  org/arangodb/graph  and  org/arangodb/graph-blueprint  are deprecated. Please use module  org/arangodb/general-graph  instead.

The HTTP REST API  _api/graph  and all its methods are deprecated. Please use the general graph API  _api/gharial  instead.

Removed features

The following replication-related JavaScript methods became obsolete in ArangoDB 2.2 and have been removed in ArangoDB 2.4:

- require("org/arangodb/replication").logger.start()

- require("org/arangodb/replication").logger.stop()

- require("org/arangodb/replication").logger.properties()

The REST API methods for these functions have also been removed in ArangoDB 2.4:

- HTTP PUT  /_api/replication/logger-start

- HTTP PUT  /_api/replication/logger-stop

- HTTP GET  /_api/replication/logger-config

- HTTP PUT  /_api/replication/logger-config

Client applications that call one of these methods should be adjusted by removing the calls to these methods. This shouldn't be problematic as these methods have been no-ops since ArangoDB 2.2 anyway.

Upgrading to ArangoDB 2.4

Please read the following sections if you upgrade from a previous version to ArangoDB 2.4. Please be sure that you have checked the list of changes in 2.4 before upgrading.

Please note first that a database directory used with ArangoDB 2.4 cannot be used with earlier versions (e.g. ArangoDB 2.3) any more. Upgrading a database directory cannot be reverted. Therefore please make sure to create a full backup of your existing ArangoDB installation before performing an upgrade.

Database Directory Version Check and Upgrade

ArangoDB will perform a database version check at startup. When ArangoDB 2.4 encounters a database created with earlier versions of ArangoDB, it will refuse to start. This is intentional.

The output will then look like this:

2014-12-22T12:02:28Z [12001] ERROR In database '_system': Database directory version (20302) is lower than current version (20400).

2014-12-22T12:02:28Z [12001] ERROR In database '_system': ----------------------------------------------------------------------

2014-12-22T12:02:28Z [12001] ERROR In database '_system': It seems like you have upgraded the ArangoDB binary.

2014-12-22T12:02:28Z [12001] ERROR In database '_system': If this is what you wanted to do, please restart with the

2014-12-22T12:02:28Z [12001] ERROR In database '_system':   --upgrade

2014-12-22T12:02:28Z [12001] ERROR In database '_system': option to upgrade the data in the database directory.

2014-12-22T12:02:28Z [12001] ERROR In database '_system': Normally you can use the control script to upgrade your database

2014-12-22T12:02:28Z [12001] ERROR In database '_system':   /etc/init.d/arangodb stop

2014-12-22T12:02:28Z [12001] ERROR In database '_system':   /etc/init.d/arangodb upgrade

2014-12-22T12:02:28Z [12001] ERROR In database '_system':   /etc/init.d/arangodb start

2014-12-22T12:02:28Z [12001] ERROR In database '_system': ----------------------------------------------------------------------

2014-12-22T12:02:28Z [12001] FATAL Database '_system' needs upgrade. Please start the server with the --upgrade option

To make ArangoDB 2.4 start with a database directory created with an earlier ArangoDB version, you may need to invoke the upgrade procedure once. This can be done by running ArangoDB from the command line and supplying the  --upgrade  option:

unix> arangod data --upgrade

where  data  is ArangoDB's main data directory.

Note: here the same database should be specified that is also specified when arangod is started regularly. Please do not run the  --upgrade  command on each individual database subfolder (named  database-<some number> ).

For example, if you regularly start your ArangoDB server with

unix> arangod mydatabasefolder

then running

unix> arangod mydatabasefolder --upgrade

will perform the upgrade for the whole ArangoDB instance, including all of its databases.

Starting with  --upgrade  will run a database version check and perform any necessary migrations. As usual, you should create a backup of your database directory before performing the upgrade.

The last line of the output should look like this:

2014-12-22T12:03:31Z [12026] INFO database upgrade passed

Please check the full output the  --upgrade  run. Upgrading may produce errors, which need to be fixed before ArangoDB can be used properly. If no errors are present or they have been resolved manually, you can start ArangoDB 2.4 regularly.

Upgrading a cluster planned in the web interface

A cluster of ArangoDB instances has to be upgraded as well. This involves upgrading all ArangoDB instances in the cluster, as well as running the version check on the whole running cluster in the end.

We have tried to make this procedure as painless and convenient for you. We assume that you planned, launched and administrated a cluster using the graphical front end in your browser. The upgrade procedure is then as follows:

1. First shut down your cluster using the graphical front end as usual.

2. Then upgrade all dispatcher instances on all machines in your cluster using the version check as described above and restart them.

3. Now open the cluster dash board in your browser by pointing it to the same dispatcher that you used to plan and launch the cluster in the graphical front end. In addition to the usual buttons "Relaunch", "Edit cluster plan" and "Delete cluster plan" you will see another button marked "Upgrade and relaunch cluster".

4. Hit this button, your cluster will be upgraded and launched and all is done for you behind the scenes. If all goes well, you will see the usual cluster dash board after a few seconds. If there is an error, you have to inspect the log files of your cluster ArangoDB instances. Please let us know if you run into problems.

There is an alternative way using the  ArangoDB  shell. Instead of steps 3. and 4. above you can launch  arangosh , point it to the dispatcher that you have used to plan and launch the cluster using the option  --server.endpoint , and execute

arangosh> require("org/arangodb/cluster").Upgrade("root","");

This upgrades the cluster and launches it, exactly as with the button above in the graphical front end. You have to replace  "root"  with a user name and  ""  with a password that is valid for authentication with the cluster.

Incompatible changes in ArangoDB 2.3

It is recommended to check the following list of incompatible changes before upgrading to ArangoDB 2.3, and adjust any client programs if necessary.

Configuration file changes

Threads and contexts

The number of server threads specified is now the minimum of threads started. There are situation in which threads are waiting for results of distributed database servers. In this case the number of threads is dynamically increased.

With ArangoDB 2.3, the number of server threads can be configured independently of the number of V8 contexts. The configuration option  --javascript.v8-contexts  was added to arangod to provide better control over the number of V8 contexts created in arangod.

Previously, the number of V8 contexts arangod created at startup was equal to the number of server threads (as specified by option  --server.threads ).

In some situations it may be more sensible to create different amounts of threads and V8 contexts. This is because each V8 contexts created will consume memory and requires CPU resources for periodic garbage collection. Contrary, server threads do not have such high memory or CPU footprint.

If the option  --javascript.v8-contexts  is not specified, the number of V8 contexts created at startup will remain equal to the number of server threads. Thus no change in configuration is required to keep the same behavior as in previous ArangoDB versions.

If you are using the default config files or merge them with your local config files, please review if the default number of server threads is okay in your environment. Additionally you should verify that the number of V8 contexts created (as specified in option  --javascript.v8-contexts ) is okay.

Syslog

The command-line option  --log.syslog  was used in previous versions of ArangoDB to turn logging to syslog on or off: when setting to a non-empty string, syslog logging was turned on, otherwise turned off. When syslog logging was turned on, logging was done with the application name specified in  --log.application , which defaulted to  triagens . There was also a command-line option  --log.hostname  which could be set but did not have any effect.

This behavior turned out to be unintuitive and was changed in 2.3 as follows:

- the command-line option  --log.syslog  is deprecated and does not have any effect when starting ArangoDB.

- to turn on syslog logging in 2.3, the option  --log.facility  has to be set to a non-empty string. The value for  facility  is OS-dependent (possible values can be found in  /usr/include/syslog.h  or the like -  user  should be available on many systems).

- the default value for  --log.application  has been changed from  triagens  to  arangod .

- the command-line option  --log.hostname  is deprecated and does not have any effect when starting ArangoDB. Instead, the host name will be set by syslog automatically.

- when logging to syslog, ArangoDB now omits the datetime prefix and the process id, because they'll be added by syslog automatically.

AQL

AQL queries throw less exceptions

ArangoDB 2.3 contains a completely rewritten AQL query optimizer and execution engine. This means that AQL queries will be executed with a different engine than in ArangoDB 2.2 and earlier. Parts of AQL queries might be executed in different order than before because the AQL optimizer has more freedom to move things around in a query.

In previous versions of ArangoDB, AQL queries aborted with an exception in many situations and threw a runtime exception. Exceptions were thrown when trying to find a value using the  IN  operator in a non-array element, when trying to use non-boolean values with the logical operands  &&  or  ||  or  ! , when using non-numeric values in arithmetic operations, when passing wrong parameters into functions etc.

In ArangoDB 2.3 this has been changed in many cases to make AQL more user-friendly and to allow the optimization to perform much more query optimizations.

Here is a summary of changes:

- when a non-array value is used on the right-hand side of the  IN  operator, the result will be  false  in ArangoDB 2.3, and no exception will be thrown.

- the boolean operators  &&  and  ||  do not throw in ArangoDB 2.3 if any of the operands is not a boolean value. Instead, they will perform an implicit cast of the values to booleans. Their result will be as follows:

- lhs && rhs  will return  lhs  if it is  false  or would be  false  when converted into a boolean. If  lhs  is  true  or would be  true  when converted to a boolean,  rhs  will be returned.

- lhs || rhs  will return  lhs  if it is  true  or would be  true  when converted into a boolean. If  lhs  is  false  or would be  false  when converted to a boolean,  rhs  will be returned.

- ! value  will return the negated value of  value  converted into a boolean

- the arithmetic operators ( + ,  - ,  * ,  / ,  % ) can be applied to any value and will not throw exceptions when applied to non-numeric values. Instead, any value used in these operators will be casted to a numeric value implicitly. If no numeric result can be produced by an arithmetic operator, it will return  null  in ArangoDB 2.3. This is also true for division by zero.

- passing arguments of invalid types into AQL functions does not throw a runtime exception in most cases, but may produce runtime warnings. Built-in AQL functions that receive invalid arguments will then return  null .

Nested FOR loop execution order

The query optimizer in 2.3 may permute the order of nested  FOR  loops in AQL queries, provided that exchanging the loops will not alter a query result. However, a change in the order of returned values is allowed because no sort order is guaranteed by AQL (and was never) unless an explicit  SORT  statement is used in a query.

Changed return values of ArangoQueryCursor.getExtra()

The return value of  ArangoQueryCursor.getExtra()  has been changed in ArangoDB 2.3. It now contains a  stats  attribute with statistics about the query previously executed. It also contains a  warnings  attribute with warnings that happened during query execution. The return value structure has been unified in 2.3 for both read-only and data-modification queries.

The return value looks like this for a read-only query:

arangosh> stmt = db._createStatement("FOR i IN mycollection RETURN i"); stmt.execute().getExtra()

{

"stats" : {

"writesExecuted" : 0,

"writesIgnored" : 0,

"scannedFull" : 2600,

"scannedIndex" : 0

},

"warnings" : [ ]

}

For data-modification queries, ArangoDB 2.3 returns a result with the same structure:

arangosh> stmt = db._createStatement("FOR i IN xx REMOVE i IN xx"); stmt.execute().getExtra()

{

"stats" : {

"writesExecuted" : 2600,

"writesIgnored" : 0,

"scannedFull" : 2600,

"scannedIndex" : 0

},

"warnings" : [ ]

}

In ArangoDB 2.2, the return value of  ArangoQueryCursor.getExtra()  was empty for read-only queries and contained an attribute  operations  with two sub-attributes for data-modification queries:

arangosh> stmt = db._createStatement("FOR i IN mycollection RETURN i"); stmt.execute().getExtra()

{

}

arangosh> stmt = db._createStatement("FOR i IN mycollection REMOVE i IN mycollection"); stmt.execute().getExtra()

{

"operations" : {

"executed" : 2600,

"ignored" : 0

}

}

Changed return values in HTTP method  POST /_api/cursor

The previously mentioned change also leads to the statistics being returned in the HTTP REST API method  POST /_api/cursor . Previously, the return value contained an optional  extra  attribute that was filled only for data-modification queries and in some other cases as follows:

{

"result" : [ ],

"hasMore" : false,

"extra" : {

"operations" : {

"executed" : 2600,

"ignored" : 0

}

}

}

With the changed result structure in ArangoDB 2.3, the  extra  attribute in the result will look like this:

{

"result" : [],

"hasMore" : false,

"extra" : {

"stats" : {

"writesExecuted" : 2600,

"writesIgnored" : 0,

"scannedFull" : 0,

"scannedIndex" : 0

},

"warnings" : [ ]

}

}

If the query option  fullCount  is requested, the  fullCount  result value will also be returned inside the  stats  attribute of the  extra  attribute, and not directly as an attribute inside the  extra  attribute as in 2.2. Note that a  fullCount  will only be present in  extra . stats  if it was requested as an option for the query.

The result in ArangoDB 2.3 will also contain a  warnings  attribute with the array of warnings that happened during query execution.

Changed return values in ArangoStatement.explain()

The return value of  ArangoStatement.explain()  has changed significantly in ArangoDB 2.3. The new return value structure is not compatible with the structure returned by 2.2.

In ArangoDB 2.3, the full execution plan for an AQL query is returned alongside all applied optimizer rules, optimization warnings etc. It is also possible to have the optimizer return all execution plans. This required a new data structure.

Client programs that use  ArangoStatement.explain()  or the HTTP REST API method  POST /_api/explain  may need to be adjusted to use the new return format.

The return value of  ArangoStatement.parse()  has been extended in ArangoDB 2.3. In addition to the existing attributes, ArangoDB 2.3 will also return an  ast  attribute containing the abstract syntax tree of the statement. This extra attribute can safely be ignored by client programs.

Variables not updatable in queries

Previous versions of ArangoDB allowed the modification of variables inside AQL queries, e.g.

LET counter = 0

FOR i IN 1..10

LET counter = counter + 1

RETURN counter

While this is admittedly a convenient feature, the new query optimizer design did not allow to keep it. Additionally, updating variables inside a query would prevent a lot of optimizations to queries that we would like the optimizer to make. Additionally, updating variables in queries that run on different nodes in a cluster would like cause non-deterministic behavior because queries are not executed linearly.

Changed return value of  TO_BOOL

The AQL function  TO_BOOL  now always returns true if its argument is a array or an object. In previous versions of ArangoDB, the function returned false for empty arrays or for objects without attributes.

Changed return value of  TO_NUMBER

The AQL function  TO_NUMBER  now returns null if its argument is an object or an array with more than one member. In previous version of ArangoDB, the return value in these cases was 0.  TO_NUMBER  will return 0 for empty array, and the numeric equivalent of the array member's value for arrays with a single member.

New AQL keywords

The following keywords have been added to AQL in ArangoDB 2.3:

- NOT

- AND

- OR

Unquoted usage of these keywords for attribute names in AQL queries will likely fail in ArangoDB 2.3. If any such attribute name needs to be used in a query, it should be enclosed in backticks to indicate the usage of a literal attribute name.

Removed features

Bitarray indexes

Bitarray indexes were only half-way documented and integrated in previous versions of ArangoDB so their benefit was limited. The support for bitarray indexes has thus been removed in ArangoDB 2.3. It is not possible to create indexes of type "bitarray" with ArangoDB 2.3.

When a collection is opened that contains a bitarray index definition created with a previous version of ArangoDB, ArangoDB will ignore it and log the following warning:

index type 'bitarray' is not supported in this version of ArangoDB and is ignored

Future versions of ArangoDB may automatically remove such index definitions so the warnings will eventually disappear.

Other removed features

The HTTP REST API method at  POST /_admin/modules/flush  has been removed.

Known issues

In ArangoDB 2.3.0, AQL queries containing filter conditions with an IN expression will not yet use an index:

FOR doc IN collection FILTER doc.indexedAttribute IN [ ... ] RETURN doc

FOR doc IN collection

FILTER doc.indexedAttribute IN [ ... ]

RETURN doc

We’re currently working on getting the IN optimizations done, and will ship them in a 2.3 maintenance release soon (e.g. 2.3.1 or 2.3.2).

Upgrading to ArangoDB 2.3

Please read the following sections if you upgrade from a previous version to ArangoDB 2.3. Please be sure that you have checked the list of changes in 2.3 before upgrading.

Please note first that a database directory used with ArangoDB 2.3 cannot be used with earlier versions (e.g. ArangoDB 2.2) any more. Upgrading a database directory cannot be reverted. Therefore please make sure to create a full backup of your existing ArangoDB installation before performing an upgrade.

Database Directory Version Check and Upgrade

ArangoDB will perform a database version check at startup. When ArangoDB 2.3 encounters a database created with earlier versions of ArangoDB, it will refuse to start. This is intentional.

The output will then look like this:

2014-11-03T15:48:06Z [2694] ERROR In database '_system': Database directory version (2.2) is lower than current version (20300).

2014-11-03T15:48:06Z [2694] ERROR In database '_system': ----------------------------------------------------------------------

2014-11-03T15:48:06Z [2694] ERROR In database '_system': It seems like you have upgraded the ArangoDB binary.

2014-11-03T15:48:06Z [2694] ERROR In database '_system': If this is what you wanted to do, please restart with the

2014-11-03T15:48:06Z [2694] ERROR In database '_system':   --upgrade

2014-11-03T15:48:06Z [2694] ERROR In database '_system': option to upgrade the data in the database directory.

2014-11-03T15:48:06Z [2694] ERROR In database '_system': Normally you can use the control script to upgrade your database

2014-11-03T15:48:06Z [2694] ERROR In database '_system':   /etc/init.d/arangodb stop

2014-11-03T15:48:06Z [2694] ERROR In database '_system':   /etc/init.d/arangodb upgrade

2014-11-03T15:48:06Z [2694] ERROR In database '_system':   /etc/init.d/arangodb start

2014-11-03T15:48:06Z [2694] ERROR In database '_system': ----------------------------------------------------------------------

2014-11-03T15:48:06Z [2694] FATAL Database '_system' needs upgrade. Please start the server with the --upgrade option

To make ArangoDB 2.3 start with a database directory created with an earlier ArangoDB version, you may need to invoke the upgrade procedure once. This can be done by running ArangoDB from the command line and supplying the  --upgrade  option:

unix> arangod data --upgrade

where  data  is ArangoDB's main data directory.

Note: here the same database should be specified that is also specified when arangod is started regularly. Please do not run the  --upgrade  command on each individual database subfolder (named  database-<some number> ).

For example, if you regularly start your ArangoDB server with

unix> arangod mydatabasefolder

then running

unix> arangod mydatabasefolder --upgrade

will perform the upgrade for the whole ArangoDB instance, including all of its databases.

Starting with  --upgrade  will run a database version check and perform any necessary migrations. As usual, you should create a backup of your database directory before performing the upgrade.

The output should look like this:

2014-11-03T15:48:47Z [2708] INFO In database '_system': Found 24 defined task(s), 5 task(s) to run

2014-11-03T15:48:47Z [2708] INFO In database '_system': state prod/standalone/upgrade, tasks updateUserModel, createStatistics, upgradeClusterPlan, setupQueues, setupJobs

2014-11-03T15:48:48Z [2708] INFO In database '_system': upgrade successfully finished

2014-11-03T15:48:48Z [2708] INFO database upgrade passed

Please check the output the  --upgrade  run. It may produce errors, which need to be fixed before ArangoDB can be used properly. If no errors are present or they have been resolved, you can start ArangoDB 2.3 regularly.

Upgrading a cluster planned in the web interface

A cluster of ArangoDB instances has to be upgraded as well. This involves upgrading all ArangoDB instances in the cluster, as well as running the version check on the whole running cluster in the end.

We have tried to make this procedure as painless and convenient for you. We assume that you planned, launched and administrated a cluster using the graphical front end in your browser. The upgrade procedure is then as follows:

1. First shut down your cluster using the graphical front end as usual.

2. Then upgrade all dispatcher instances on all machines in your cluster using the version check as described above and restart them.

3. Now open the cluster dash board in your browser by pointing it to the same dispatcher that you used to plan and launch the cluster in the graphical front end. In addition to the usual buttons "Relaunch", "Edit cluster plan" and "Delete cluster plan" you will see another button marked "Upgrade and relaunch cluster".

4. Hit this button, your cluster will be upgraded and launched and all is done for you behind the scenes. If all goes well, you will see the usual cluster dash board after a few seconds. If there is an error, you have to inspect the log files of your cluster ArangoDB instances. Please let us know if you run into problems.

There is an alternative way using the  ArangoDB  shell. Instead of steps 3. and 4. above you can launch  arangosh , point it to the dispatcher that you have used to plan and launch the cluster using the option  --server.endpoint , and execute

arangosh> require("org/arangodb/cluster").Upgrade("root","");

This upgrades the cluster and launches it, exactly as with the button above in the graphical front end. You have to replace  "root"  with a user name and  ""  with a password that is valid for authentication with the cluster.

Upgrading to ArangoDB 2.2

Please read the following sections if you upgrade from a previous version to ArangoDB 2.2.

Please note first that a database directory used with ArangoDB 2.2 cannot be used with earlier versions (e.g. ArangoDB 2.1) any more. Upgrading a database directory cannot be reverted. Therefore please make sure to create a full backup of your existing ArangoDB installation before performing an upgrade.

Database Directory Version Check and Upgrade

ArangoDB will perform a database version check at startup. When ArangoDB 2.2 encounters a database created with earlier versions of ArangoDB, it will refuse to start. This is intentional.

The output will then look like this:

2014-07-07T22:04:53Z [18675] ERROR In database '_system': Database directory version (2.1) is lower than server version (2.2).

2014-07-07T22:04:53Z [18675] ERROR In database '_system': ----------------------------------------------------------------------

2014-07-07T22:04:53Z [18675] ERROR In database '_system': It seems like you have upgraded the ArangoDB binary.

2014-07-07T22:04:53Z [18675] ERROR In database '_system': If this is what you wanted to do, please restart with the

2014-07-07T22:04:53Z [18675] ERROR In database '_system':   --upgrade

2014-07-07T22:04:53Z [18675] ERROR In database '_system': option to upgrade the data in the database directory.

2014-07-07T22:04:53Z [18675] ERROR In database '_system': Normally you can use the control script to upgrade your database

2014-07-07T22:04:53Z [18675] ERROR In database '_system':   /etc/init.d/arangodb stop

2014-07-07T22:04:53Z [18675] ERROR In database '_system':   /etc/init.d/arangodb upgrade

2014-07-07T22:04:53Z [18675] ERROR In database '_system':   /etc/init.d/arangodb start

2014-07-07T22:04:53Z [18675] ERROR In database '_system': ----------------------------------------------------------------------

2014-07-07T22:04:53Z [18675] FATAL Database version check failed for '_system'. Please start the server with the --upgrade option

To make ArangoDB 2.2 start with a database directory created with an earlier ArangoDB version, you may need to invoke the upgrade procedure once. This can be done by running ArangoDB from the command line and supplying the  --upgrade  option:

unix> arangod data --upgrade

where  data  is ArangoDB's main data directory.

Note: here the same database should be specified that is also specified when arangod is started regularly. Please do not run the  --upgrade  command on each individual database subfolder (named  database-<some number> ).

For example, if you regularly start your ArangoDB server with

unix> arangod mydatabasefolder

then running

unix> arangod mydatabasefolder --upgrade

will perform the upgrade for the whole ArangoDB instance, including all of its databases.

Starting with  --upgrade  will run a database version check and perform any necessary migrations. As usual, you should create a backup of your database directory before performing the upgrade.

The output should look like this:

2014-07-07T22:11:30Z [18867] INFO In database '_system': starting upgrade from version 2.1 to 2.2.0

2014-07-07T22:11:30Z [18867] INFO In database '_system': Found 19 defined task(s), 2 task(s) to run

2014-07-07T22:11:30Z [18867] INFO In database '_system': upgrade successfully finished

2014-07-07T22:11:30Z [18867] INFO database upgrade passed

Please check the output the  --upgrade  run. It may produce errors, which need to be fixed before ArangoDB can be used properly. If no errors are present or they have been resolved, you can start ArangoDB 2.2 regularly.

Upgrading a cluster planned in the web interface

A cluster of ArangoDB instances has to be upgraded as well. This involves upgrading all ArangoDB instances in the cluster, as well as running the version check on the whole running cluster in the end.

We have tried to make this procedure as painless and convenient for you. We assume that you planned, launched and administrated a cluster using the graphical front end in your browser. The upgrade procedure is then as follows:

1. First shut down your cluster using the graphical front end as usual.

2. Then upgrade all dispatcher instances on all machines in your cluster using the version check as described above and restart them.

3. Now open the cluster dash board in your browser by pointing it to the same dispatcher that you used to plan and launch the cluster in the graphical front end. In addition to the usual buttons "Relaunch", "Edit cluster plan" and "Delete cluster plan" you will see another button marked "Upgrade and relaunch cluster".

4. Hit this button, your cluster will be upgraded and launched and all is done for you behind the scenes. If all goes well, you will see the usual cluster dash board after a few seconds. If there is an error, you have to inspect the log files of your cluster ArangoDB instances. Please let us know if you run into problems.

There is an alternative way using the  ArangoDB  shell. Instead of steps 3. and 4. above you can launch  arangosh , point it to the dispatcher that you have used to plan and launch the cluster using the option  --server.endpoint , and execute

arangosh> require("org/arangodb/cluster").Upgrade("root","");

This upgrades the cluster and launches it, exactly as with the button above in the graphical front end. You have to replace  "root"  with a user name and  ""  with a password that is valid for authentication with the cluster.

Set up your ArangoDB Cluster

Setting up a cluster can be intimidating task. You have to deal with firewalls, ports, different types of machines, and the like. ArangoDB is prepared to deal with all kinds of different setups and requirements.

However, in the following section we concentrate on a standard setup and show you, how to build a ArangoDB cluster within minutes. If you want to dive deeper into the nasty details, you should read about Sharding.

Development Scenario

While not really relevant for a production environment, a common setup for development is to create a cluster on a single machine. This is the easiest of all setups and you should be ready to play with a ArangoDB cluster in less than a minute. Even when developing it is convenient to create a cluster on a single machine instead of having to deal with a lot of servers.

Step 1: Enable the Cluster mode

In order to enable the cluster mode, edit the configuration file as root

vi /etc/arangodb/arangod.conf

and change the lines

[cluster]

disable-dispatcher-kickstarter = yes

disable-dispatcher-frontend = yes

to

[cluster]

disable-dispatcher-kickstarter = no

disable-dispatcher-frontend = no

Save and restart

/etc/init.d/arangodb restart

Step 2: Setup your Cluster

You can now configure your cluster. A cluster consists of a number of database server and coordinators. A database server holds your precious data, while a coordinator talks to the outside world, takes requests from clients, distributes them to the database server, and assembles the result.

For this example, we assume that are creating three database servers and two coordinators.

Use your favorite web browser and go to

http://localhost:8529/

You will now see the cluster management frontend.

Select Single Machine scenario. The next page allows you to enter the number of coordinators and database servers.

Press Launch Cluster to fire up the cluster. That's it. Your cluster is up and running.

Step 3: Test your Cluster

Click on one of the coordinators (e. g. "Claus") to access your cluster. In order to create a sharded collection, use Tools / JS Shell and execute

JSH> db._create("users", { numberOfShards: 3 });

JSH> db.users.save({ _key: "cmeier", firstName: "Claus", lastName: "Meier" });

Congratulations! You have created your first sharded collection and stored a document in it.

Features and Improvements

The following list shows in detail which features have been added or improved in ArangoDB 2.8. ArangoDB 2.8 also contains several bugfixes that are not listed here. For a list of bugfixes, please consult the CHANGELOG.

AQL improvements

AQL Graph Traversals / Pattern Matching

AQL offers a new feature to traverse over a graph without writing JavaScript functions but with all the other features you know from AQL. For this purpose, a special version of  FOR variable-name IN expression  has been introduced.

This special version has the following format:  FOR vertex-variable, edge-variable, path-variable IN traversal-expression , where  traversal-expression  has the following format:  [depth] direction start-vertex graph-definition  with the following input parameters:

- depth (optional): defines how many steps are executed. The value can either be an integer value (e.g.  3 ) or a range of integer values (e.g.  1..5 ). The default is 1.

- direction: defines which edge directions are followed. Can be either  OUTBOUND ,  INBOUND  or  ANY .

- start-vertex: defines where the traversal is started. Must be an  _id  value or a document.

- graph-definition: defines which edge collections are used for the traversal. Must be either  GRAPH graph-name  for graphs created with the graph-module, or a list of edge collections  edge-col1, edge-col2, .. edge-colN .

The three output variables have the following semantics:

- vertex-variable: The last visited vertex.

- edge-variable: The last visited edge (optional).

- path-variable: The complete path from start-vertex to vertex-variable (optional).

The traversal statement can be used in the same way as the original  FOR variable-name IN expression , and can be combined with filters and other AQL constructs.

As an example one can now find the friends of a friend for a certain user with this AQL statement:

FOR foaf, e, path IN 2 ANY @startUser GRAPH "relations"

FILTER path.edges[0].type == "friend"

FILTER path.edges[1].type == "friend"

FILTER foaf._id != @startUser

RETURN DISTINCT foaf

Optimizer rules have been implemented to gain performance of the traversal statement. These rules move filter statements into the traversal statement s.t. paths which can never pass the filter are not emitted to the variables.

As an example take the query above and assume there are edges that do not have  type == "friend" . If in the first edge step there is such a non-friend edge the second steps will never be computed for these edges as they cannot fulfill the filter condition.

Array Indexes

Hash indexes and skiplist indexes can now optionally be defined for array values so that they index individual array members instead of the entire array value.

To define an index for array values, the attribute name is extended with the expansion operator  [*]  in the index definition.

Example:

db._create("posts");

db.posts.ensureHashIndex("tags[*]");

When given the following document

{

"tags": [

"AQL",

"ArangoDB",

"Index"

]

}

this index will now contain the individual values  "AQL" ,  "ArangoDB"  and  "Index" .

Now the index can be used for finding all documents having  "ArangoDB"  somewhere in their  tags  array using the following AQL query:

FOR doc IN posts

FILTER "ArangoDB" IN doc.tags[*]

RETURN doc

It is also possible to create an index on sub-attributes of array values. This makes sense when the index attribute is an array of objects, e.g.

db._drop("posts");

db._create("posts");

db.posts.ensureIndex({ type: "hash", fields: [ "tags[*].name" ] });

db.posts.insert({ tags: [ { name: "AQL" }, { name: "ArangoDB" }, { name: "Index" } ] });

db.posts.insert({ tags: [ { name: "AQL" }, { name: "2.8" } ] });

The following query will then use the array index:

FOR doc IN posts

FILTER 'AQL' IN doc.tags[*].name

RETURN doc

Array values will automatically be de-duplicated before being inserted into an array index.

Please note that filtering using array indexes only works from within AQL queries and only if the query filters on the indexed attribute using the  IN  operator. The other comparison operators ( == ,  != ,  > ,  >= ,  < ,  <= ) currently do not use array indexes.

Optimizer improvements

The AQL query optimizer can now use indexes if multiple filter conditions on attributes of the same collection are combined with logical ORs, and if the usage of indexes would completely cover these conditions.

For example, the following queries can now use two independent indexes on  value1  and  value2  (the latter query requires that the indexes are skiplist indexes due to usage of the  <  and  >  comparison operators):

FOR doc IN collection FILTER doc.value1 == 42 || doc.value2 == 23 RETURN doc

FOR doc IN collection FILTER doc.value1 < 42 || doc.value2 > 23 RETURN doc

The new optimizer rule "sort-in-values" can now pre-sort the right-hand side operand of  IN  and  NOT IN  operators so the operation can use a binary search with logarithmic complexity instead of a linear search. The rule will be applied when the right-hand side operand of an  IN  or  NOT IN  operator in a filter condition is a variable that is defined in a different loop/scope than the operator itself. Additionally, the filter condition must consist of solely the  IN  or  NOT IN  operation in order to avoid any side-effects.

The rule will kick in for a queries such as the following:

LET values = /* some runtime expression here */

FOR doc IN collection

FILTER doc.value IN values

RETURN doc

It will not be applied for the followig queries, because the right-hand side operand of the  IN  is either not a variable, or because the FILTER condition may have side effects:

FOR doc IN collection

FILTER doc.value IN /* some runtime expression here */

RETURN doc

LET values = /* some runtime expression here */

FOR doc IN collection

FILTER FUNCTION(doc.values) == 23 && doc.value IN values

RETURN doc

AQL functions added

The following AQL functions have been added in 2.8:

- POW(base, exponent) : returns the base to the exponent exp

- UNSET_RECURSIVE(document, attributename, ...) : recursively removes the attributes attributename (can be one or many) from document and its sub-documents. All other attributes will be preserved. Multiple attribute names can be specified by either passing multiple individual string argument names, or by passing an array of attribute names:

UNSET_RECURSIVE(doc, '_id', '_key', 'foo', 'bar')

UNSET_RECURSIVE(doc, [ '_id', '_key', 'foo', 'bar' ])

- IS_DATESTRING(value) : returns true if value is a string that can be used in a date function. This includes partial dates such as 2015 or 2015-10 and strings containing invalid dates such as 2015-02-31. The function will return false for all non-string values, even if some of them may be usable in date functions.

Miscellaneous improvements

- the ArangoShell now provides the convenience function  db._explain(query)  for retrieving a human-readable explanation of AQL queries. This function is a shorthand for  require("org/arangodb/aql/explainer").explain(query) .

- the AQL query optimizer now automatically converts  LENGTH(collection-name)  to an optimized expression that returns the number of documents in a collection. Previous versions of ArangoDB returned a warning when using this expression and also enumerated all documents in the collection, which was inefficient.

- improved performance of skipping over many documents in an AQL query when no indexes and no filters are used, e.g.

FOR doc IN collection

LIMIT 1000000, 10

RETURN doc

- added cluster execution site info in execution plan explain output for AQL queries

- for 30+ AQL functions there is now an additional implementation in C++ that removes the need for internal data conversion when the function is called

- the AQL editor in the web interface now supports using bind parameters

Deadlock detection

ArangoDB 2.8 now has an automatic deadlock detection for transactions.

A deadlock is a situation in which two or more concurrent operations (user transactions or AQL queries) try to access the same resources (collections, documents) and need to wait for the others to finish, but none of them can make any progress.

In case of such a deadlock, there would be no progress for any of the involved transactions, and none of the involved transactions could ever complete. This is completely undesirable, so the new automatic deadlock detection mechanism in ArangoDB will automatically kick in and abort one of the transactions involved in such a deadlock. Aborting means that all changes done by the transaction will be rolled back and error 29 ( deadlock detected ) will be thrown.

Client code (AQL queries, user transactions) that accesses more than one collection should be aware of the potential of deadlocks and should handle the error 29 ( deadlock detected ) properly, either by passing the exception to the caller or retrying the operation.

Replication

The following improvements for replication have been made in 2.8 (note: most of them have been backported to ArangoDB 2.7 as well):

- added  autoResync  configuration parameter for continuous replication.

When set to  true , a replication slave will automatically trigger a full data re-synchronization with the master when the master cannot provide the log data the slave had asked for. Note that  autoResync  will only work when the option  requireFromPresent  is also set to  true  for the continuous replication, or when the continuous syncer is started and detects that no start tick is present.

Automatic re-synchronization may transfer a lot of data from the master to the slave and may be expensive. It is therefore turned off by default. When turned off, the slave will never perform an automatic re-synchronization with the master.

- added  idleMinWaitTime  and  idleMaxWaitTime  configuration parameters for continuous replication.

These parameters can be used to control the minimum and maximum wait time the slave will (intentionally) idle and not poll for master log changes in case the master had sent the full logs already. The  idleMaxWaitTime  value will only be used when  adapativePolling  is set to  true . When  adaptivePolling  is disabled, only  idleMinWaitTime  will be used as a constant time span in which the slave will not poll the master for further changes. The default values are 0.5 seconds for  idleMinWaitTime  and 2.5 seconds for  idleMaxWaitTime , which correspond to the hard-coded values used in previous versions of ArangoDB.

- added  initialSyncMaxWaitTime  configuration parameter for initial and continuous replication

This option controls the maximum wait time (in seconds) that the initial synchronization will wait for a response from the master when fetching initial collection data. If no response is received within this time period, the initial synchronization will give up and fail. This option is also relevant for continuous replication in case autoResync is set to true, as then the continuous replication may trigger a full data re-synchronization in case the master cannot the log data the slave had asked for.

- HTTP requests sent from the slave to the master during initial synchronization will now be retried if they fail with connection problems.

- the initial synchronization now logs its progress so it can be queried using the regular replication status check APIs.

- added  async  attribute for  sync  and  syncCollection  operations called from the ArangoShell. Setthing this attribute to  true  will make the synchronization job on the server go into the background, so that the shell does not block. The status of the started asynchronous synchronization job can be queried from the ArangoShell like this:

/* starts initial synchronization */

var replication = require("org/arangodb/replication");

var id = replication.sync({

endpoint: "tcp://master.domain.org:8529",

username: "myuser",

password: "mypasswd",

async: true

});

/* now query the id of the returned async job and print the status */

print(replication.getSyncResult(id));

The result of  getSyncResult()  will be  false  while the server-side job has not completed, and different to  false  if it has completed. When it has completed, all job result details will be returned by the call to  getSyncResult() .

- the web admin interface dashboard now shows a server's replication status at the bottom of the page

Web Admin Interface

The following improvements have been made for the web admin interface:

- the AQL editor now has support for bind parameters. The bind parameter values can be edited in the web interface and saved with a query for future use.

- the AQL editor now allows canceling running queries. This can be used to cancel long-running queries without switching to the query management section.

- the dashboard now provides information about the server's replication status at the bottom of the page. This can be used to track either the status of a one-time synchronization or the continuous replication.

- the compaction status and some status internals about collections are now displayed in the detail view for a collection in the web interface. These data can be used for debugging compaction issues.

- unloading a collection via the web interface will now trigger garbage collection in all v8 contexts and force a WAL flush. This increases the chances of perfoming the unload faster.

- the status terminology for collections for which an unload request has been issued via the web interface was changed from  in the process of being unloaded  to  will be unloaded . This is more accurate as the actual unload may be postponed until later if there are still references pointing to data in the collection.

Foxx improvements

- the module resolution used by  require  now behaves more like in node.js

- the  org/arangodb/request  module now returns response bodies for error responses by default. The old behavior of not returning bodies for error responses can be re-enabled by explicitly setting the option  returnBodyOnError  to  false

Miscellaneous changes

The startup option  --server.hide-product-header  can be used to make the server not send the HTTP response header  "Server: ArangoDB"  in its HTTP responses. This can be used to conceal the server make from HTTP clients. By default, the option is turned off so the header is still sent as usual.

arangodump and arangorestore now have better error reporting. Additionally, arangodump will now fail by default when trying to dump edges that refer to already dropped collections. This can be circumvented by specifying the option  --force true  when invoking arangodump.

arangoimp now provides an option  --create-collection-type  to specify the type of the collection to be created when  --create-collection  is set to  true . Previously  --create-collection  always created document collections and the creation of edge collections was not possible.

Features and Improvements

The following list shows in detail which features have been added or improved in ArangoDB 2.7. ArangoDB 2.7 also contains several bugfixes that are not listed here. For a list of bugfixes, please consult the CHANGELOG.

Performance improvements

Index buckets

The primary indexes and hash indexes of collections can now be split into multiple index buckets. This option was available for edge indexes only in ArangoDB 2.6.

A bucket can be considered a container for a specific range of index values. For primary, hash and edge indexes, determining the responsible bucket for an index value is done by hashing the actual index value and applying a simple arithmetic operation on the hash.

Because an index value will be present in at most one bucket and buckets are independent, using multiple buckets provides the following benefits:

- initially building the in-memory index data can be parallelized even for a single index, with one thread per bucket (or with threads being responsible for more than one bucket at a time). This can help reducing the loading time for collections.

- resizing an index when it is about to run out of reserve space is performed per bucket. As each bucket only contains a fraction of the entire index, resizing and rehashing a bucket is much faster and less intrusive than resizing and rehashing the entire index.

When creating new collections, the default number of index buckets is  8  since ArangoDB 2.7. In previous versions, the default value was  1 . The number of buckets can also be adjusted for existing collections so they can benefit from the optimizations. The number of index buckets can be set for a collection at any time by using a collection's  properties  function:

db.collection.properties({ indexBuckets: 16 });

The number of index buckets must be a power of 2.

Please note that for building the index data for multiple buckets in parallel it is required that a collection contains a significant amount of documents because for a low number of documents the overhead of parallelization will outweigh its benefits. The current threshold value is 256k documents, but this value may change in future versions of ArangoDB. Additionally, the configuration option  --database.index-threads  will determine how many parallel threads may be used for building the index data.

Faster update and remove operations in non-unique hash indexes

The unique hash indexes in ArangoDB provided an amortized O(1) lookup, insert, update and remove performance. Non-unique hash indexes provided amortized O(1) insert performance, but had worse performance for update and remove operations for non-unique values. For documents with the same index value, they maintained a list of collisions. When a document was updated or removed, that exact document had to be found in the collisions list for the index value. While getting to the start of the collisions list was O(1), scanning the list had O(n) performance in the worst case (with n being the number of documents with the same index value). Overall, this made update and remove operations in non-unique hash indexes slow if the index contained many duplicate values.

This has been changed in ArangoDB 2.7 so that non-unique hash indexes now also provide update and remove operations with an amortized complexity of O(1), even if there are many duplicates.

Resizing non-unique hash indexes now also doesn't require looking into the document data (which may involve a disk access) because the index maintains some internal cache value per document. When resizing and rehashing the index (or an index bucket), the index will first compare only the cache values before peeking into the actual documents. This change can also lead to reduced index resizing times.

Throughput enhancements

The ArangoDB-internal implementations for dispatching requests, keeping statistics and assigning V8 contexts to threads have been improved in order to use less locks. These changes allow higher concurrency and throughput in these components, which can also make the server handle more requests in a given period of time.

What gains can be expected depends on which operations are executed, but there are real-world cases in which throughput increased by between 25 % and 70 % when compared to 2.6.

Madvise hints

The Linux variant for ArangoDB provides the OS with madvise hints about index memory and datafile memory. These hints can speed up things when memory is tight, in particular at collection load time but also for random accesses later. There is no formal guarantee that the OS actually uses the madvise hints provided by ArangoDB, but actual measurements have shown improvements for loading bigger collections.

AQL improvements

Additional date functions

ArangoDB 2.7 provides several extra AQL functions for date and time calculation and manipulation. These functions were contributed by GitHub users @CoDEmanX and @friday. A big thanks for their work!

The following extra date functions are available from 2.7 on:

- DATE_DAYOFYEAR(date) : Returns the day of year number of date. The return values range from 1 to 365, or 366 in a leap year respectively.

- DATE_ISOWEEK(date) : Returns the ISO week date of date. The return values range from 1 to 53. Monday is considered the first day of the week. There are no fractional weeks, thus the last days in December may belong to the first week of the next year, and the first days in January may be part of the previous year's last week.

- DATE_LEAPYEAR(date) : Returns whether the year of date is a leap year.

- DATE_QUARTER(date) : Returns the quarter of the given date (1-based):

- 1: January, February, March

- 2: April, May, June

- 3: July, August, September

- 4: October, November, December

- DATE_DAYS_IN_MONTH(date): Returns the number of days in date's month (28..31).

- DATE_ADD(date, amount, unit) : Adds amount given in unit to date and returns the calculated date.

unit can be either of the following to specify the time unit to add or subtract (case-insensitive):

- y, year, years

- m, month, months

- w, week, weeks

- d, day, days

- h, hour, hours

- i, minute, minutes

- s, second, seconds

- f, millisecond, milliseconds

amount is the number of units to add (positive value) or subtract (negative value).

- DATE_SUBTRACT(date, amount, unit) : Subtracts amount given in unit from date and returns the calculated date.

It works the same as  DATE_ADD() , except that it subtracts. It is equivalent to calling  DATE_ADD()  with a negative amount, except that  DATE_SUBTRACT()  can also subtract ISO durations. Note that negative ISO durations are not supported (i.e. starting with  -P , like  -P1Y ).

- DATE_DIFF(date1, date2, unit, asFloat) : Calculate the difference between two dates in given time unit, optionally with decimal places. Returns a negative value if date1 is greater than date2.

- DATE_COMPARE(date1, date2, unitRangeStart, unitRangeEnd) : Compare two partial dates and return true if they match, false otherwise. The parts to compare are defined by a range of time units.

The full range is: years, months, days, hours, minutes, seconds, milliseconds. Pass the unit to start from as unitRangeStart, and the unit to end with as unitRangeEnd. All units in between will be compared. Leave out unitRangeEnd to only compare unitRangeStart.

- DATE_FORMAT(date, format) : Format a date according to the given format string. It supports the following placeholders (case-insensitive):

- %t: timestamp, in milliseconds since midnight 1970-01-01

- %z: ISO date (0000-00-00T00:00:00.000Z)

- %w: day of week (0..6)

- %y: year (0..9999)

- %yy: year (00..99), abbreviated (last two digits)

- %yyyy: year (0000..9999), padded to length of 4

- %yyyyyy: year (-009999 .. +009999), with sign prefix and padded to length of 6

- %m: month (1..12)

- %mm: month (01..12), padded to length of 2

- %d: day (1..31)

- %dd: day (01..31), padded to length of 2

- %h: hour (0..23)

- %hh: hour (00..23), padded to length of 2

- %i: minute (0..59)

- %ii: minute (00..59), padded to length of 2

- %s: second (0..59)

- %ss: second (00..59), padded to length of 2

- %f: millisecond (0..999)

- %fff: millisecond (000..999), padded to length of 3

- %x: day of year (1..366)

- %xxx: day of year (001..366), padded to length of 3

- %k: ISO week date (1..53)

- %kk: ISO week date (01..53), padded to length of 2

- %l: leap year (0 or 1)

- %q: quarter (1..4)

- %a: days in month (28..31)

- %mmm: abbreviated English name of month (Jan..Dec)

- %mmmm: English name of month (January..December)

- %www: abbreviated English name of weekday (Sun..Sat)

- %wwww: English name of weekday (Sunday..Saturday)

- %&: special escape sequence for rare occasions

- %%: literal %

- %: ignored

RETURN DISTINCT

To return unique values from a query, AQL now provides the  DISTINCT  keyword. It can be used as a modifier for  RETURN  statements, as a shorter alternative to the already existing  COLLECT  statement.

For example, the following query only returns distinct (unique)  status  attribute values from the collection:

FOR doc IN collection

RETURN DISTINCT doc.status

RETURN DISTINCT  is not allowed on the top-level of a query if there is no  FOR  loop in front of it.  RETURN DISTINCT  is allowed in subqueries.

RETURN DISTINCT  ensures that the values returned are distinct (unique), but does not guarantee any order of results. In order to have certain result order, an additional  SORT  statement must be added to a query.

Shorthand object notation

AQL now provides a shorthand notation for object literals in the style of ES6 object literals:

LET name = "Peter"

LET age = 42

RETURN { name, age }

This is equivalent to the previously available canonical form, which is still available and supported:

LET name = "Peter"

LET age = 42

RETURN { name : name, age : age }

Array expansion improvements

The already existing [\*] operator has been improved with optional filtering and projection and limit capabilities.

For example, consider the following example query that filters values from an array attribute:

FOR u IN users

RETURN {

name: u.name,

friends: (

FOR f IN u.friends

FILTER f.age > u.age

RETURN f.name

)

}

With the [\*] operator, this query can be simplified to

FOR u IN users

RETURN { name: u.name, friends: u.friends[* FILTER CURRENT.age > u.age].name }

The pseudo-variable CURRENT can be used to access the current array element. The  FILTER  condition can refer to  CURRENT  or any variables valid in the outer scope.

To return a projection of the current element, there can now be an inline  RETURN :

FOR u IN users

RETURN u.friends[* RETURN CONCAT(CURRENT.name, " is a friend of ", u.name)]

which is the simplified variant for:

FOR u IN users

RETURN (

FOR friend IN u.friends

RETURN CONCAT(friend.name, " is a friend of ", u.name)

)

Array contraction

In order to collapse (or flatten) results in nested arrays, AQL now provides the [\*\*] operator. It works similar to the [\*] operator, but additionally collapses nested arrays. How many levels are collapsed is determined by the amount of \* characters used.

For example, consider the following query that produces a nested result:

FOR u IN users

RETURN u.friends[*].name

The [\*\*] operator can now be applied to get rid of the nested array and turn it into a flat array. We simply apply the [\*\*] on the previous query result:

RETURN (

FOR u IN users RETURN u.friends[*].name

)[**]

Template query strings

Assembling query strings in JavaScript has been error-prone when using simple string concatenation, especially because plain JavaScript strings do not have multiline-support, and because of potential parameter injection issues. While multiline query strings can be assembled with ES6 template strings since ArangoDB 2.5, and query bind parameters are there since ArangoDB 1.0 to prevent parameter injection, there was no JavaScript-y solution to combine these.

ArangoDB 2.7 now provides an ES6 template string generator function that can be used to easily and safely assemble AQL queries from JavaScript. JavaScript variables and expressions can be used easily using regular ES6 template string substitutions:

let name = 'test';

let attributeName = '_key';

let query = aqlQuery`FOR u IN users

FILTER u.name == ${name}

RETURN u.${attributeName}`;

db._query(query);

This is more legible than when using a plain JavaScript string and also does not require defining the bind parameter values separately:

let name = 'test';

let attributeName = '_key';

let query = "FOR u IN users " +

"FILTER u.name == @name " +

"RETURN u.@attributeName";

db._query(query, {

name,

attributeName

});

The  aqlQuery  template string generator will also handle collection objects automatically:

db._query(aqlQuery`FOR u IN ${ db.users } RETURN u.name`);

Note that while template strings are available in the JavaScript functions provided to build queries, they aren't a feature of AQL itself. AQL could always handle multiline query strings and provided bind parameters ( @... ) for separating the query string and the parameter values. The  aqlQuery  template string generator function will take care of this separation, too, but will do it behind the scenes.

AQL query result cache

The AQL query result cache can optionally cache the complete results of all or just selected AQL queries. It can be operated in the following modes:

- off : the cache is disabled. No query results will be stored

- on : the cache will store the results of all AQL queries unless their  cache  attribute flag is set to  false

- demand : the cache will store the results of AQL queries that have their  cache  attribute set to  true , but will ignore all others

The mode can be set at server startup using the  --database.query-cache-mode  configuration option and later changed at runtime. The default value is  off , meaning that the query result cache is disabled. This is because the cache may consume additional memory to keep query results, and also because it must be invalidated when changes happen in collections for which results have been cached.

The query result cache may therefore have positive or negative effects on query execution times, depending on the workload: it will not make much sense turning on the cache in write-only or write-mostly scenarios, but the cache may be very beneficial in case workloads are read-only or read-mostly, and query are complex.

If the query cache is operated in  demand  mode, it can be controlled per query if the cache should be checked for a result.

Miscellaneous changes

Optimizer

The AQL optimizer rule  patch-update-statements  has been added. This rule can optimize certain AQL UPDATE queries that update documents in the a collection that they also iterate over.

For example, the following query reads documents from a collection in order to update them:

FOR doc IN collection

UPDATE doc WITH { newValue: doc.oldValue + 1 } IN collection

In this case, only a single collection is affected by the query, and there is no index lookup involved to find the to-be-updated documents. In this case, the UPDATE query does not require taking a full, memory-intensive snapshot of the collection, but it can be performed in small chunks. This can lead to memory savings when executing such queries.

Function call arguments optimization

This optimization will lead to arguments in function calls inside AQL queries not being copied but being passed by reference. This may speed up calls to functions with bigger argument values or queries that call AQL functions a lot of times.

Web Admin Interface

The web interface now has a new design.

The "Applications" tab in the web interfaces has been renamed to "Services".

The ArangoDB API documentation has been moved from the "Tools" menu to the "Links" menu. The new documentation is based on Swagger 2.0 and opens in a separate web page.

Foxx improvements

Confidential configuration

It is now possible to specify configuration options with the type  password . The password type is equivalent to the text type but will be masked in the web frontend to prevent accidental exposure of confidential options like API keys and passwords when configuring your Foxx application.

Dependencies

The syntax for specifying dependencies in manifests has been extended to allow specifying optional dependencies. Unmet optional dependencies will not prevent an app from being mounted. The traditional shorthand syntax for specifying non-optional dependencies will still be supported in the upcoming versions of ArangoDB.

Before:

{

...

"dependencies": {

"notReallyNeeded": "users:^1.0.0",

"totallyNecessary": "sessions:^1.0.0"

}

}

After:

{

"dependencies": {

"notReallyNeeded": {

"name": "users",

"version": "^1.0.0",

"required": false

},

"totallyNecessary": {

"name": "sessions",

"version": "^1.0.0"

}

}

}

Replication

The existing replication HTTP API has been extended with methods that replication clients can use to determine whether a given date, identified by a tick value, is still present on a master for replication. By calling these APIs, clients can make an informed decision about whether the master can still provide all missing data starting from the point up to which the client had already synchronized. This can be helpful in case a replication client is re-started after a pause.

Master servers now also track up the point up to which they have sent changes to clients for replication. This information can be used to determine the point of data that replication clients have received from the master, and if and how far approximately they lag behind.

Finally, restarting the replication applier on a slave server has been made more robust in case the applier was stopped while there were pending transactions on the master server, and re-starting the replication applier needs to restore the state of these transactions.

Client tools

The filenames in dumps created by arangodump now contain not only the name of the dumped collection, but also an additional 32-digit hash value. This is done to prevent overwriting dump files in case-insensitive file systems when there exist multiple collections with the same name (but with different cases).

For example, if a database had two collections test and Test, previous versions of arangodump created the following files:

- test.structure.json  and  test.data.json  for collection test

- Test.structure.json  and  Test.data.json  for collection Test

This did not work in case-insensitive filesystems, because the files for the second collection would have overwritten the files of the first. arangodump in 2.7 will create the unique files in this case, by appending the 32-digit hash value to the collection name in all case. These filenames will be unambiguous even in case-insensitive filesystems.

Miscellaneous changes

Better control-C support in arangosh

When CTRL-C is pressed in arangosh, it will now abort the locally running command (if any). If no command was running, pressing CTRL-C will print a  ^C  first. Pressing CTRL-C again will then quit arangosh.

CTRL-C can also be used to reset the current prompt while entering complex nested objects which span multiple input lines.

CTRL-C support has been added to the ArangoShell versions built with Readline-support (Linux and MacOS only). The Windows version of ArangoDB uses a different library for handling input, and support for CTRL-C has not been added there yet.

Start / stop

Linux startup scripts and systemd configuration for arangod now try to adjust the NOFILE (number of open files) limits for the process. The limit value is set to 131072 (128k) when ArangoDB is started via start/stop commands.

This will prevent arangod running out of available file descriptors in case of many parallel HTTP connections or large collections with many datafiles.

Additionally, when ArangoDB is started/stopped manually via the start/stop commands, the main process will wait for up to 10 seconds after it forks the supervisor and arangod child processes. If the startup fails within that period, the start/stop script will fail with a non-zero exit code, allowing any invoking scripts to handle this error. Previous versions always returned an exit code of 0, even when arangod couldn't be started.

If the startup of the supervisor or arangod is still ongoing after 10 seconds, the main program will still return with exit code 0 in order to not block any scripts. The limit of 10 seconds is arbitrary because the time required for an arangod startup is not known in advance.

Non-sparse logfiles

WAL logfiles and datafiles created by arangod are now non-sparse. This prevents SIGBUS signals being raised when a memory-mapped region backed by a sparse datafile was accessed and the memory region was not actually backed by disk, for example because the disk ran out of space.

arangod now always fully allocates the disk space required for a logfile or datafile when it creates one, so the memory region can always be backed by disk, and memory can be accessed without SIGBUS being raised.

Features and Improvements

The following list shows in detail which features have been added or improved in ArangoDB 2.6. ArangoDB 2.6 also contains several bugfixes that are not listed here. For a list of bugfixes, please consult the CHANGELOG.

APIs added

Batch document removal and lookup commands

The following commands have been added for  collection  objects:

- collection.lookupByKeys(keys)

- collection.removeByKeys(keys)

These commands can be used to perform multi-document lookup and removal operations efficiently from the ArangoShell. The argument to these operations is an array of document keys.

These commands can also be used via the HTTP REST API. Their endpoints are:

- PUT /_api/simple/lookup-by-keys

- PUT /_api/simple/remove-by-keys

Collection export HTTP REST API

ArangoDB now provides a dedicated collection export API, which can take snapshots of entire collections more efficiently than the general-purpose cursor API. The export API is useful to transfer the contents of an entire collection to a client application. It provides optional filtering on specific attributes.

The export API is available at endpoint  POST /_api/export?collection=... . The API has the same return value structure as the already established cursor API ( POST /_api/cursor ).

An introduction to the export API is given in this blog post: http://jsteemann.github.io/blog/2015/04/04/more-efficient-data-exports/

AQL improvements

EDGES AQL Function

The AQL function EDGES got a new fifth optional parameter, which must be an object if specified. Right now only one option is available for it:

- includeVertices  this is a boolean parameter that allows to modify the result of  EDGES() . The default value for  includeVertices  is  false , which does not have any effect. Setting it to  true  will modify the result, such that also the connected vertices are returned along with the edges:

{ vertex: <vertexDocument>, edge: <edgeDocument> }

Subquery optimizations for AQL queries

This optimization avoids copying intermediate results into subqueries that are not required by the subquery.

A brief description can be found here: http://jsteemann.github.io/blog/2015/05/04/subquery-optimizations/

Return value optimization for AQL queries

This optimization avoids copying the final query result inside the query's main  ReturnNode .

A brief description can be found here: http://jsteemann.github.io/blog/2015/05/04/return-value-optimization-for-aql/

Speed up AQL queries containing big  IN  lists for index lookups

IN  lists used for index lookups had performance issues in previous versions of ArangoDB. These issues have been addressed in 2.6 so using bigger  IN  lists for filtering is much faster.

A brief description can be found here: http://jsteemann.github.io/blog/2015/05/07/in-list-improvements/

Added alternative implementation for AQL COLLECT

The alternative method uses a hash table for grouping and does not require its input elements to be sorted. It will be taken into account by the optimizer for  COLLECT  statements that do not use an  INTO  clause.

In case a  COLLECT  statement can use the hash table variant, the optimizer will create an extra plan for it at the beginning of the planning phase. In this plan, no extra  SORT  node will be added in front of the  COLLECT  because the hash table variant of  COLLECT  does not require sorted input. Instead, a  SORT  node will be added after it to sort its output. This  SORT  node may be optimized away again in later stages. If the sort order of the result is irrelevant to the user, adding an extra  SORT null  after a hash  COLLECT  operation will allow the optimizer to remove the sorts altogether.

In addition to the hash table variant of  COLLECT , the optimizer will modify the original plan to use the regular  COLLECT  implementation. As this implementation requires sorted input, the optimizer will insert a  SORT  node in front of the  COLLECT . This  SORT  node may be optimized away in later stages.

The created plans will then be shipped through the regular optimization pipeline. In the end, the optimizer will pick the plan with the lowest estimated total cost as usual. The hash table variant does not require an up-front sort of the input, and will thus be preferred over the regular  COLLECT  if the optimizer estimates many input elements for the  COLLECT  node and cannot use an index to sort them.

The optimizer can be explicitly told to use the regular sorted variant of  COLLECT  by suffixing a  COLLECT  statement with  OPTIONS { "method" : "sorted" } . This will override the optimizer guesswork and only produce the sorted variant of  COLLECT .

A blog post on the new  COLLECT  implementation can be found here: http://jsteemann.github.io/blog/2015/04/22/collecting-with-a-hash-table/

Simplified return value syntax for data-modification AQL queries

ArangoDB 2.4 since version allows to return results from data-modification AQL queries. The syntax for this was quite limited and verbose:

FOR i IN 1..10

INSERT { value: i } IN test

LET inserted = NEW

RETURN inserted

The  LET inserted = NEW RETURN inserted  was required literally to return the inserted documents. No calculations could be made using the inserted documents.

This is now more flexible. After a data-modification clause (e.g.  INSERT ,  UPDATE ,  REPLACE ,  REMOVE ,  UPSERT ) there can follow any number of  LET  calculations. These calculations can refer to the pseudo-values  OLD  and  NEW  that are created by the data-modification statements.

This allows returning projections of inserted or updated documents, e.g.:

FOR i IN 1..10

INSERT { value: i } IN test

RETURN { _key: NEW._key, value: i }

Still not every construct is allowed after a data-modification clause. For example, no functions can be called that may access documents.

More information can be found here: http://jsteemann.github.io/blog/2015/03/27/improvements-for-data-modification-queries/

Added AQL  UPSERT  statement

This adds an  UPSERT  statement to AQL that is a combination of both  INSERT  and  UPDATE  /  REPLACE . The  UPSERT  will search for a matching document using a user-provided example. If no document matches the example, the insert part of the  UPSERT  statement will be executed. If there is a match, the update / replace part will be carried out:

UPSERT { page: 'index.html' }               /* search example */

INSERT { page: 'index.html', pageViews: 1 } /* insert part */

UPDATE { pageViews: OLD.pageViews + 1 }     /* update part */

IN pageViews

UPSERT  can be used with an  UPDATE  or  REPLACE  clause. The  UPDATE  clause will perform a partial update of the found document, whereas the  REPLACE  clause will replace the found document entirely. The  UPDATE  or  REPLACE  parts can refer to the pseudo-value  OLD , which contains all attributes of the found document.

UPSERT  statements can optionally return values. In the following query, the return attribute  found  will return the found document before the  UPDATE  was applied. If no document was found,  found  will contain a value of  null . The  updated  result attribute will contain the inserted / updated document:

UPSERT { page: 'index.html' }               /* search example */

INSERT { page: 'index.html', pageViews: 1 } /* insert part */

UPDATE { pageViews: OLD.pageViews + 1 }     /* update part */

IN pageViews

RETURN { found: OLD, updated: NEW }

A more detailed description of  UPSERT  can be found here: http://jsteemann.github.io/blog/2015/03/27/preview-of-the-upsert-command/

Miscellaneous changes

When errors occur inside AQL user functions, the error message will now contain a stacktrace, indicating the line of code in which the error occurred. This should make debugging AQL user functions easier.

Web Admin Interface

ArangoDB's built-in web interface now uses sessions. Session information is stored in cookies, so clients using the web interface must accept cookies in order to use it.

The new startup option  --server.session-timeout  can be used for adjusting the session lifetime.

The AQL editor in the web interface now provides an explain functionality, which can be used for inspecting and performance-tuning AQL queries. The query execution time is now also displayed in the AQL editor.

Foxx apps that require configuration or are missing dependencies are now indicated in the app overview and details.

Foxx improvements

Configuration and Dependencies

Foxx app manifests can now define configuration options, as well as dependencies on other Foxx apps.

An introduction to Foxx configurations can be found in the blog: https://www.arangodb.com/2015/05/reusable-foxx-apps-with-configurations/

And the blog post on Foxx dependencies can be found here: https://www.arangodb.com/2015/05/foxx-dependencies-for-more-composable-foxx-apps/

For more information on manifests see the Manifest chapter. More information on using dependencies can be found in the chapter on Foxx exports.

Mocha Tests

You can now write tests for your Foxx apps using the Mocha testing framework: https://www.arangodb.com/2015/04/testing-foxx-mocha/

A recipe for writing tests for your Foxx apps can be found in the cookbook: https://docs.arangodb.com/cookbook/FoxxTesting.html

API Documentation

The API documentation has been updated to Swagger 2. You can now also mount API documentation in your own Foxx apps. For more information see the API Documentation chapter in the Foxx documentation.

Also see the blog post introducing this feature: https://www.arangodb.com/2015/05/document-your-foxx-apps-with-swagger-2/

Custom Scripts and Foxx Queue

In addition to the existing setup and teardown scripts you can now define custom scripts in your Foxx manifest and invoke these using the web admin interface or the Foxx manager CLI. These scripts can now also take positional arguments and export return values.

Job types for the Foxx Queue can now be defined as a script name and app mount path allowing the use of Foxx scripts as job types. The pre-2.6 job types are known to cause issues when restarting the server and are error-prone; we strongly recommended converting any existing job types to the new format.

Client tools

The default configuration value for the option  --server.request-timeout  was increased from 300 to 1200 seconds for all client tools (arangosh, arangoimp, arangodump, arangorestore).

The default configuration value for the option  --server.connect-timeout  was increased from 3 to 5 seconds for client tools (arangosh, arangoimp, arangodump, arangorestore).

Arangorestore

The option  --create-database  was added for arangorestore.

Setting this option to  true  will now create the target database if it does not exist. When creating the target database, the username and passwords passed to arangorestore will be used to create an initial user for the new database.

The default value for this option is  false .

Arangoimp

Arangoimp can now optionally update or replace existing documents, provided the import data contains documents with  _key  attributes.

Previously, the import could be used for inserting new documents only, and re-inserting a document with an existing key would have failed with a unique key constraint violated error.

The behavior of arangoimp (insert, update, replace on duplicate key) can now be controlled with the option  --on-duplicate . The option can have one of the following values:

- error : when a unique key constraint error occurs, do not import or update the document but report an error. This is the default.

- update : when a unique key constraint error occurs, try to (partially) update the existing document with the data specified in the import. This may still fail if the document would violate secondary unique indexes. Only the attributes present in the import data will be updated and other attributes already present will be preserved. The number of updated documents will be reported in the  updated  attribute of the HTTP API result.

- replace : when a unique key constraint error occurs, try to fully replace the existing document with the data specified in the import. This may still fail if the document would violate secondary unique indexes. The number of replaced documents will be reported in the  updated  attribute of the HTTP API result.

- ignore : when a unique key constraint error occurs, ignore this error. There will be no insert, update or replace for the particular document. Ignored documents will be reported separately in the  ignored  attribute of the HTTP API result.

The default value is  error .

A few examples for using arangoimp with the  --on-duplicate  option can be found here: http://jsteemann.github.io/blog/2015/04/14/updating-documents-with-arangoimp/

Miscellaneous changes

- Some Linux-based ArangoDB packages are now using tcmalloc for memory allocator.

- Upgraded ICU library to version 54. This increases performance in many places.

- Allow to split an edge index into buckets which are resized individually. The default value is  1 , resembling the pre-2.6 behavior. Using multiple buckets will lead to the index entries being distributed to the individual buckets, with each bucket being responsible only for a fraction of the total index entries. Using multiple buckets may lead to more frequent but much faster index bucket resizes, and is recommended for bigger edge collections.

- Default configuration value for option  --server.backlog-size  was changed from 10 to 64.

- Default configuration value for option  --database.ignore-datafile-errors  was changed from  true  to  false

- Document keys can now contain  @  and  .  characters

- Fulltext index can now index text values contained in direct sub-objects of the indexed attribute.

Previous versions of ArangoDB only indexed the attribute value if it was a string. Sub-attributes of the index attribute were ignored when fulltext indexing.

Now, if the index attribute value is an object, the object's values will each be included in the fulltext index if they are strings. If the index attribute value is an array, the array's values will each be included in the fulltext index if they are strings.

For example, with a fulltext index present on the  translations  attribute, the following text values will now be indexed:

var c = db._create("example");

c.ensureFulltextIndex("translations");

c.insert({ translations: { en: "fox", de: "Fuchs", fr: "renard", ru: "лиса" } });

c.insert({ translations: "Fox is the English translation of the German word Fuchs" });

c.insert({ translations: [ "ArangoDB", "document", "database", "Foxx" ] });

c.fulltext("translations", "лиса").toArray();       // returns only first document

c.fulltext("translations", "Fox").toArray();        // returns first and second documents

c.fulltext("translations", "prefix:Fox").toArray(); // returns all three documents

- Added configuration option  --server.foxx-queues-poll-interval

This startup option controls the frequency with which the Foxx queues manager is checking the queue (or queues) for jobs to be executed.

The default value is  1  second. Lowering this value will result in the queue manager waking up and checking the queues more frequently, which may increase CPU usage of the server. When not using Foxx queues, this value can be raised to save some CPU time.

- Added configuration option  --server.foxx-queues

This startup option controls whether the Foxx queue manager will check queue and job entries in the  _system  database only. Restricting the Foxx queue manager to the  _system  database will lead to the queue manager having to check only the queues collection of a single database, whereas making it check the queues of all databases might result in more work to be done and more CPU time to be used by the queue manager.

Features and Improvements

The following list shows in detail which features have been added or improved in ArangoDB 2.5. ArangoDB 2.5 also contains several bugfixes that are not listed here. For a list of bugfixes, please consult the CHANGELOG.

V8 version upgrade

The built-in version of V8 has been upgraded from 3.29.54 to 3.31.74.1. This allows activating additional ES6 (also dubbed Harmony or ES.next) features in ArangoDB, both in the ArangoShell and the ArangoDB server. They can be used for scripting and in server-side actions such as Foxx routes, traversals etc.

The following additional ES6 features become available in ArangoDB 2.5 by default:

- iterators and generators

- template strings

- enhanced object literals

- enhanced numeric literals

- block scoping with  let  and constant variables using  const  (note: constant variables require using strict mode, too)

- additional string methods (such as  startsWith ,  repeat  etc.)

Index improvements

Sparse hash and skiplist indexes

Hash and skiplist indexes can optionally be made sparse. Sparse indexes exclude documents in which at least one of the index attributes is either not set or has a value of  null .

As such documents are excluded from sparse indexes, they may contain fewer documents than their non-sparse counterparts. This enables faster indexing and can lead to reduced memory usage in case the indexed attribute does occur only in some, but not all documents of the collection. Sparse indexes will also reduce the number of collisions in non-unique hash indexes in case non-existing or optional attributes are indexed.

In order to create a sparse index, an object with the attribute  sparse  can be added to the index creation commands:

db.collection.ensureHashIndex(attributeName, { sparse: true });

db.collection.ensureHashIndex(attributeName1, attributeName2, { sparse: true });

db.collection.ensureUniqueConstraint(attributeName, { sparse: true });

db.collection.ensureUniqueConstraint(attributeName1, attributeName2, { sparse: true });

db.collection.ensureSkiplist(attributeName, { sparse: true });

db.collection.ensureSkiplist(attributeName1, attributeName2, { sparse: true });

db.collection.ensureUniqueSkiplist(attributeName, { sparse: true });

db.collection.ensureUniqueSkiplist(attributeName1, attributeName2, { sparse: true });

Note that in place of the above specialized index creation commands, it is recommended to use the more general index creation command  ensureIndex :

db.collection.ensureIndex({ type: "hash", sparse: true, unique: true, fields: [ attributeName ] });

db.collection.ensureIndex({ type: "skiplist", sparse: false, unique: false, fields: [ "a", "b" ] });

When not explicitly set, the  sparse  attribute defaults to  false  for new hash or skiplist indexes.

This causes a change in behavior when creating a unique hash index without specifying the sparse flag: in 2.4, unique hash indexes were implicitly sparse, always excluding  null  values. There was no option to control this behavior, and sparsity was neither supported for non-unique hash indexes nor skiplists in 2.4. This implicit sparsity of unique hash indexes was considered an inconsistency, and therefore the behavior was cleaned up in 2.5. As of 2.5, indexes will only be created sparse if sparsity is explicitly requested. Existing unique hash indexes from 2.4 or before will automatically be migrated so they are still sparse after the upgrade to 2.5.

Geo indexes are implicitly sparse, meaning documents without the indexed location attribute or containing invalid location coordinate values will be excluded from the index automatically. This is also a change when compared to pre-2.5 behavior, when documents with missing or invalid coordinate values may have caused errors on insertion when the geo index'  unique  flag was set and its  ignoreNull  flag was not. This was confusing and has been rectified in 2.5. The method  ensureGeoConstaint()  now does the same as  ensureGeoIndex() . Furthermore, the attributes  constraint ,  unique ,  ignoreNull  and  sparse  flags are now completely ignored when creating geo indexes.

The same is true for fulltext indexes. There is no need to specify non-uniqueness or sparsity for geo or fulltext indexes.

As sparse indexes may exclude some documents, they cannot be used for every type of query. Sparse hash indexes cannot be used to find documents for which at least one of the indexed attributes has a value of  null . For example, the following AQL query cannot use a sparse index, even if one was created on attribute  attr :

FOR doc In collection

FILTER doc.attr == null

RETURN doc

If the lookup value is non-constant, a sparse index may or may not be used, depending on the other types of conditions in the query. If the optimizer can safely determine that the lookup value cannot be  null , a sparse index may be used. When uncertain, the optimizer will not make use of a sparse index in a query in order to produce correct results.

For example, the following queries cannot use a sparse index on  attr  because the optimizer will not know beforehand whether the comparison values for  doc.attr  will include  null :

FOR doc In collection

FILTER doc.attr == SOME_FUNCTION(...)

RETURN doc

FOR other IN otherCollection

FOR doc In collection

FILTER doc.attr == other.attr

RETURN doc

Sparse skiplist indexes can be used for sorting if the optimizer can safely detect that the index range does not include  null  for any of the index attributes.

Selectivity estimates

Indexes of type  primary ,  edge  and  hash  now provide selectivity estimates. These will be used by the AQL query optimizer when deciding about index usage. Using selectivity estimates can lead to faster query execution when more selective indexes are used.

The selectivity estimates are also returned by the  GET /_api/index  REST API method in a sub-attribute  selectivityEstimate  for each index that supports it. This attribute will be omitted for indexes that do not provide selectivity estimates. If provided, the selectivity estimate will be a numeric value between 0 and 1.

Selectivity estimates will also be reported in the result of  collection.getIndexes()  for all indexes that support this. If no selectivity estimate can be determined for an index, the attribute  selectivityEstimate  will be omitted here, too.

The web interface also shows selectivity estimates for each index that supports this.

Currently the following index types can provide selectivity estimates:

- primary index

- edge index

- hash index (unique and non-unique)

No selectivity estimates will be provided for indexes when running in cluster mode.

AQL Optimizer improvements

Sort removal

The AQL optimizer rule "use-index-for-sort" will now remove sorts also in case a non-sorted index (e.g. a hash index) is used for only equality lookups and all sort attributes are covered by the equality lookup conditions.

For example, in the following query the extra sort on  doc.value  will be optimized away provided there is an index on  doc.value ):

FOR doc IN collection

FILTER doc.value == 1

SORT doc.value

RETURN doc

The AQL optimizer rule "use-index-for-sort" now also removes sort in case the sort criteria excludes the left-most index attributes, but the left-most index attributes are used by the index for equality-only lookups.

For example, in the following query with a skiplist index on  value1 ,  value2 , the sort can be optimized away:

FOR doc IN collection

FILTER doc.value1 == 1

SORT doc.value2

RETURN doc

Constant attribute propagation

The new AQL optimizer rule  propagate-constant-attributes  will look for attributes that are equality-compared to a constant value, and will propagate the comparison value into other equality lookups. This rule will only look inside  FILTER  conditions, and insert constant values found in  FILTER s, too.

For example, the rule will insert  42  instead of  i.value  in the second  FILTER  of the following query:

FOR i IN c1

FOR j IN c2

FILTER i.value == 42

FILTER j.value == i.value

RETURN 1

Interleaved processing

The optimizer will now inspect AQL data-modification queries and detect if the query's data-modification part can run in lockstep with the data retrieval part of the query, or if the data retrieval part must be executed and completed first before the data-modification can start.

Executing both data retrieval and data-modification in lockstep allows using much smaller buffers for intermediate results, reducing the memory usage of queries. Not all queries are eligible for this optimization, and the optimizer will only apply the optimization when it can safely detect that the data-modification part of the query will not modify data to be found by the retrieval part.

Query execution statistics

The  filtered  attribute was added to AQL query execution statistics. The value of this attribute indicates how many documents were filtered by  FilterNode s in the AQL query. Note that  IndexRangeNode s can also filter documents by selecting only the required ranges from the index. The  filtered  value will not include the work done by  IndexRangeNode s, but only the work performed by  FilterNode s.

Language improvements

Dynamic attribute names in AQL object literals

This change allows using arbitrary expressions to construct attribute names in object literals specified in AQL queries. To disambiguate expressions and other unquoted attribute names, dynamic attribute names need to be enclosed in brackets ( [  and  ] ).

Example:

FOR i IN 1..100

RETURN { [ CONCAT('value-of-', i) ] : i }

AQL functions

The following AQL functions were added in 2.5:

- MD5(value) : generates an MD5 hash of  value

- SHA1(value) : generates an SHA1 hash of  value

- RANDOM_TOKEN(length) : generates a random string value of the specified length

Simplify Foxx usage

Thanks to our user feedback we learned that Foxx is a powerful, yet rather complicated concept. With 2.5 we made it less complicated while keeping all its strength. That includes a rewrite of the documentation as well as some code changes as follows:

Moved Foxx applications to a different folder.

Until 2.4 foxx apps were stored in the following folder structure:  <app-path>/databases/<dbname>/<appname>:<appversion> . This caused some trouble as apps where cached based on name and version and updates did not apply. Also the path on filesystem and the app's access URL had no relation to one another. Now the path on filesystem is identical to the URL (except the appended APP):  <app-path>/_db/<dbname>/<mointpoint>/APP

Rewrite of Foxx routing

The routing of Foxx has been exposed to major internal changes we adjusted because of user feedback. This allows us to set the development mode per mountpoint without having to change paths and hold apps at separate locations.

Foxx Development mode

The development mode used until 2.4 is gone. It has been replaced by a much more mature version. This includes the deprecation of the javascript.dev-app-path parameter, which is useless since 2.5. Instead of having two separate app directories for production and development, apps now reside in one place, which is used for production as well as for development. Apps can still be put into development mode, changing their behavior compared to production mode. Development mode apps are still reread from disk at every request, and still they ship more debug output.

This change has also made the startup options  --javascript.frontend-development-mode  and  --javascript.dev-app-path  obsolete. The former option will not have any effect when set, and the latter option is only read and used during the upgrade to 2.5 and does not have any effects later.

Foxx install process

Installing Foxx apps has been a two step process: import them into ArangoDB and mount them at a specific mountpoint. These operations have been joined together. You can install an app at one mountpoint, that's it. No fetch, mount, unmount, purge cycle anymore. The commands have been simplified to just:

- install: get your Foxx app up and running

- uninstall: shut it down and erase it from disk

Foxx error output

Until 2.4 the errors produced by Foxx were not optimal. Often, the error message was just  unable to parse manifest  and contained only an internal stack trace. In 2.5 we made major improvements there, including a much more fine grained error output that helps you debug your Foxx apps. The error message printed is now much closer to its source and should help you track it down.

Also we added the default handlers for unhandled errors in Foxx apps:

- You will get a nice internal error page whenever your Foxx app is called but was not installed due to any error

- You will get a proper error message when having an uncaught error appears in any app route

In production mode the messages above will NOT contain any information about your Foxx internals and are safe to be exposed to third party users. In development mode the messages above will contain the stacktrace (if available), making it easier for your in-house devs to track down errors in the application.

Foxx console

We added a  console  object to Foxx apps. All Foxx apps now have a console object implementing the familiar Console API in their global scope, which can be used to log diagnostic messages to the database. This console also allows to read the error output of one specific foxx.

Foxx requests

We added  org/arangodb/request  module, which provides a simple API for making HTTP requests to external services. This is enables Foxx to be directly part of a micro service architecture.

Features and Improvements

The following list shows in detail which features have been added or improved in ArangoDB 2.4. ArangoDB 2.4 also contains several bugfixes that are not listed here. For a list of bugfixes, please consult the CHANGELOG.

V8 version upgrade

The built-in version of V8 has been upgraded from 3.16.14 to 3.29.59. This activates several ES6 (also dubbed Harmony or ES.next) features in ArangoDB, both in the ArangoShell and the ArangoDB server. They can be used for scripting and in server-side actions such as Foxx routes, traversals etc.

The following ES6 features are available in ArangoDB 2.4 by default:

- iterators

- the  of  operator

- symbols

- predefined collections types (Map, Set etc.)

- typed arrays

Many other ES6 features are disabled by default, but can be made available by starting arangod or arangosh with the appropriate options:

- arrow functions

- proxies

- generators

- String, Array, and Number enhancements

- constants

- enhanced object and numeric literals

To activate all these ES6 features in arangod or arangosh, start it with the following options:

arangosh --javascript.v8-options="--harmony --harmony_generators"

More details on the available ES6 features can be found in this blog.

FoxxGenerator

ArangoDB 2.4 is shipped with FoxxGenerator, a framework for building standardized Hypermedia APIs easily. The generated APIs can be consumed with client tools that understand Siren.

Hypermedia is the simple idea that our HTTP APIs should have links between their endpoints in the same way that our web sites have links between them. FoxxGenerator is based on the idea that you can represent an API as a statechart: Every endpoint is a state and the links are the transitions between them. Using your description of states and transitions, it can then create an API for you.

The FoxxGenerator can create APIs based on a semantic description of entities and transitions. A blog series on the use cases and how to use the Foxx generator is here:

- part 1

- part 2

- part 3

A cookbook recipe for getting started with FoxxGenerator is here.

AQL improvements

Optimizer improvements

The AQL optimizer has been enhanced to use of indexes in queries in several additional cases. Filters containing the  IN  operator can now make use of indexes, and multiple OR- or AND-combined filter conditions can now also use indexes if the filter conditions refer to the same indexed attribute.

Here are a few examples of queries that can now use indexes but couldn't before:

FOR doc IN collection

FILTER doc.indexedAttribute == 1 || doc.indexedAttribute > 99

RETURN doc

FOR doc IN collection

FILTER doc.indexedAttribute IN [ 3, 42 ] || doc.indexedAttribute > 99

RETURN doc

FOR doc IN collection

FILTER (doc.indexedAttribute > 2 && doc.indexedAttribute < 10) ||

(doc.indexedAttribute > 23 && doc.indexedAttribute < 42)

RETURN doc

Additionally, the optimizer rule  remove-filter-covered-by-index  has been added. This rule removes FilterNodes and CalculationNodes from an execution plan if the filter condition is already covered by a previous IndexRangeNode. Removing the filter's CalculationNode and the FilterNode itself will speed up query execution because the query requires less computation.

Furthermore, the new optimizer rule  remove-sort-rand  will remove a  SORT RAND()  statement and move the random iteration into the appropriate  EnumerateCollectionNode . This is usually more efficient than individually enumerating and sorting.

Data-modification queries returning documents

INSERT ,  REMOVE ,  UPDATE  or  REPLACE  queries now can optionally return the documents inserted, removed, updated, or replaced. This is helpful for tracking the auto-generated attributes (e.g.  _key ,  _rev ) created by an  INSERT  and in a lot of other situations.

In order to return documents from a data-modification query, the statement must immediately be immediately followed by a  LET  statement that assigns either the pseudo-value  NEW  or  OLD  to a variable. This  LET  statement must be followed by a  RETURN  statement that returns the variable introduced by  LET :

FOR i IN 1..100

INSERT { value: i } IN test LET inserted = NEW RETURN inserted

FOR u IN users

FILTER u.status == 'deleted'

REMOVE u IN users LET removed = OLD RETURN removed

FOR u IN users

FILTER u.status == 'not active'

UPDATE u WITH { status: 'inactive' } IN users LET updated = NEW RETURN updated

NEW  refers to the inserted or modified document revision, and  OLD  refers to the document revision before update or removal.  INSERT  statements can only refer to the  NEW  pseudo-value, and  REMOVE  operations only to  OLD .  UPDATE  and  REPLACE  can refer to either.

In all cases the full documents will be returned with all their attributes, including the potentially auto-generated attributes such as  _id ,  _key , or  _rev  and the attributes not specified in the update expression of a partial update.

Language improvements

COUNT  clause

An optional  COUNT  clause was added to the  COLLECT  statement. The  COUNT  clause allows for more efficient counting of values.

In previous versions of ArangoDB one had to write the following to count documents:

RETURN LENGTH (

FOR doc IN collection

FILTER ...some condition...

RETURN doc

)

With the  COUNT  clause, the query can be modified to

FOR doc IN collection

FILTER ...some condition...

COLLECT WITH COUNT INTO length

RETURN length

The latter query will be much more efficient because it will not produce any intermediate results with need to be shipped from a subquery into the  LENGTH  function.

The  COUNT  clause can also be used to count the number of items in each group:

FOR doc IN collection

FILTER ...some condition...

COLLECT group = doc.group WITH COUNT INTO length

return { group: group, length: length }

COLLECT  modifications

In ArangoDB 2.4,  COLLECT  operations can be made more efficient if only a small fragment of the group values is needed later. For these cases,  COLLECT  provides an optional conversion expression for the  INTO  clause. This expression controls the value that is inserted into the array of group values. It can be used for projections.

The following query only copies the  dateRegistered  attribute of each document into the groups, potentially saving a lot of memory and computation time compared to copying  doc  completely:

FOR doc IN collection

FILTER ...some condition...

COLLECT group = doc.group INTO dates = doc.dateRegistered

return { group: group, maxDate: MAX(dates) }

Compare this to the following variant of the query, which was the only way to achieve the same result in previous versions of ArangoDB:

FOR doc IN collection

FILTER ...some condition...

COLLECT group = doc.group INTO dates

return { group: group, maxDate: MAX(dates[*].doc.dateRegistered) }

The above query will need to copy the full  doc  attribute into the  lengths  variable, whereas the new variant will only copy the  dateRegistered  attribute of each  doc .

Subquery syntax

In previous versions of ArangoDB, subqueries required extra parentheses around them, and this caused confusion when subqueries were used as function parameters. For example, the following query did not work:

LET values = LENGTH(

FOR doc IN collection RETURN doc

)

but had to be written as follows:

LET values = LENGTH((

FOR doc IN collection RETURN doc

))

This was unintuitive and is fixed in version 2.4 so that both variants of the query are accepted and produce the same result.

Web interface

The  Applications  tab for Foxx applications in the web interface has got a complete redesign.

It will now only show applications that are currently running in ArangoDB. For a selected application, a new detailed view has been created. This view provides a better overview of the app, e.g.:

- author

- license

- version

- contributors

- download links

- API documentation

Installing a new Foxx application on the server is made easy using the new  Add application  button. The  Add application  dialog provides all the features already available in the  foxx-manager  console application plus some more:

- install a Foxx application from Github

- install a Foxx application from a zip file

- install a Foxx application from ArangoDB's application store

- create a new Foxx application from scratch: this feature uses a generator to create a Foxx application with pre-defined CRUD methods for a given list of collections. The generated Foxx app can either be downloaded as a zip file or be installed on the server. Starting with a new Foxx app has never been easier.

Miscellaneous improvements

Default endpoint is 127.0.0.1

The default endpoint for the ArangoDB server has been changed from  0.0.0.0  to  127.0.0.1 . This will make new ArangoDB installations unaccessible from clients other than localhost unless the configuration is changed. This is a security precaution measure that has been requested as a feature a lot of times.

If you are the development option  --enable-relative , the endpoint will still be  0.0.0.0 .

System collections in replication

By default, system collections are now included in replication and all replication API return values. This will lead to user accounts and credentials data being replicated from master to slave servers. This may overwrite slave-specific database users.

If this is undesired, the  _users  collection can be excluded from replication easily by setting the  includeSystem  attribute to  false  in the following commands:

- replication.sync({ includeSystem: false });

- replication.applier.properties({ includeSystem: false });

This will exclude all system collections (including  _aqlfunctions ,  _graphs  etc.) from the initial synchronization and the continuous replication.

If this is also undesired, it is also possible to specify a list of collections to exclude from the initial synchronization and the continuous replication using the  restrictCollections  attribute, e.g.:

require("org/arangodb/replication").applier.properties({

includeSystem: true,

restrictType: "exclude",

restrictCollections: [ "_users", "_graphs", "foo" ]

});

Features and Improvements

The following list shows in detail which features have been added or improved in ArangoDB 2.3. ArangoDB 2.3 also contains several bugfixes that are not listed here.

AQL improvements

Framework improvements

AQL queries are now sent through a query optimizer framework before execution. The query optimizer framework will first convert the internal representation of the query, the abstract syntax tree, into an initial execution plan.

The execution plan is then send through optimizer rules that may directly modify the plan in place or create a new variant of the plan. New plans might again be optimized, allowing the optimizer to carry out several optimizations.

After creating plans, the optimizer will estimate the costs for each plan and pick the plan with the lowest cost (termed the optimal plan) for the actual query execution.

With the  explain()  method of  ArangoStatement  users can check which execution plan the optimizer pick or retrieve a list of other plans that optimizer did not choose. The plan will reveal many details about which indexes are used etc.  explain()  will also return the of optimizer rules applied so users can validate whether or not a query allows using a specific optimization.

Execution of AQL queries has been rewritten in C++, allowing many queries to avoid the conversion of documents between ArangoDB's internal low-level data structure and the V8 object representation format.

The framework for optimizer rules is now also generally cluster-aware, allowing specific optimizations for queries that run in a cluster. Additionally, the optimizer was designed to be extensible in order to add more optimizations in the future.

Language improvements

Alternative operator syntax

ArangoDB 2.3 allows to use the following alternative forms for the logical operators:

- AND : logical and

- OR : logical or

- NOT : negation

This new syntax is just an alternative to the old syntax, allowing easier migration from SQL. The old syntax is still fully supported and will be:

- && : logical and

- || : logical or

- ! : negation

NOT IN  operator

AQL now has a dedicated  NOT IN  operator.

Previously, a  NOT IN  was only achievable by writing a negated  IN  condition:

FOR i IN ... FILTER ! (i IN [ 23, 42 ]) ...

In ArangoDB 2.3, the same result can now alternatively be achieved by writing the more intuitive variant:

FOR i IN ... FILTER i NOT IN [ 23, 42 ] ...

Improvements of built-in functions

The following AQL string functions have been added:

- LTRIM(value, characters) : left-trims a string value

- RTRIM(value, characters) : right-trims a string value

- FIND_FIRST(value, search, start, end) : finds the first occurrence of a search string

- FIND_LAST(value, search, start, end) : finds the last occurrence of a search string

- SPLIT(value, separator, limit) : splits a string into an array, using a separator

- SUBSTITUTE(value, search, replace, limit) : replaces characters or strings inside another

The following other AQL functions have been added:

- VALUES(document) : returns the values of an object as a array (this is the counterpart to the already existing  ATTRIBUTES  function)

- ZIP(attributes, values) : returns an object constructed from attributes and values passed in separate parameters

- PERCENTILE(values, n, method) : returns the nths percentile of the values provided, using rank or interpolation method

The already existing functions  CONCAT  and  CONCAT_SEPARATOR  now support array arguments, e.g.:

/* "foobarbaz" */

CONCAT([ 'foo', 'bar', 'baz'])

/* "foo,bar,baz" */

CONCAT_SEPARATOR(", ", [ 'foo', 'bar', 'baz'])

AQL queries throw less exceptions

In previous versions of ArangoDB, AQL queries aborted with an exception in many situations and threw a runtime exception. For example, exceptions were thrown when trying to find a value using the  IN  operator in a non-array element, when trying to use non-boolean values with the logical operands  &&  or  ||  or  ! , when using non-numeric values in arithmetic operations, when passing wrong parameters into functions etc.

The fact that many AQL operators could throw exceptions led to a lot of questions from users, and a lot of more-verbose-than-necessary queries. For example, the following query failed when there were documents that did not have a  topics  attribute at all:

FOR doc IN mycollection

FILTER doc.topics IN [ "something", "whatever" ]

RETURN doc

This forced users to rewrite the query as follows:

FOR doc IN mycollection

FILTER IS_LIST(doc.topics) && doc.topics IN [ "something", "whatever" ]

RETURN doc

In ArangoDB 2.3 this has been changed to make AQL easier to use. The change provides an extra benefit, and that is that non-throwing operators allow the query optimizer to perform much more transformations in the query without changing its overall result.

Here is a summary of changes:

- when a non-array value is used on the right-hand side of the  IN  operator, the result will be  false  in ArangoDB 2.3, and no exception will be thrown.

- the boolean operators  &&  and  ||  do not throw in ArangoDB 2.3 if any of the operands is not a boolean value. Instead, they will perform an implicit cast of the values to booleans. Their result will be as follows:

- lhs && rhs  will return  lhs  if it is  false  or would be  false  when converted into a boolean. If  lhs  is  true  or would be  true  when converted to a boolean,  rhs  will be returned.

- lhs || rhs  will return  lhs  if it is  true  or would be  true  when converted into a boolean. If  lhs  is  false  or would be  false  when converted to a boolean,  rhs  will be returned.

- ! value  will return the negated value of  value  converted into a boolean

- the arithmetic operators ( + ,  - ,  * ,  / ,  % ) can be applied to any value and will not throw exceptions when applied to non-numeric values. Instead, any value used in these operators will be casted to a numeric value implicitly. If no numeric result can be produced by an arithmetic operator, it will return  null  in ArangoDB 2.3. This is also true for division by zero.

- passing arguments of invalid types into AQL functions does not throw a runtime exception in most cases, but may produce runtime warnings. Built-in AQL functions that receive invalid arguments will then return  null .

Performance improvements

Non-unique hash indexes

The performance of insertion into non-unique hash indexes has been improved significantly. This fixes performance problems in case attributes were indexes that contained only very few distinct values, or when most of the documents did not even contain the indexed attribute. This also fixes problems when loading collections with such indexes.

The insertion time now scales linearly with the number of documents regardless of the cardinality of the indexed attribute.

Reverse iteration over skiplist indexes

AQL queries can now use a sorted skiplist index for reverse iteration. This allows several queries to run faster than in previous versions of ArangoDB.

For example, the following AQL query can now use the index on  doc.value :

FOR doc IN mycollection

FILTER doc.value > 23

SORT doc.values DESC

RETURN doc

Previous versions of ArangoDB did not use the index because of the descending ( DESC ) sort.

Additionally, the new AQL optimizer can use an index for sorting now even if the AQL query does not contain a  FILTER  statement. This optimization was not available in previous versions of ArangoDB.

Added basic support for handling binary data in Foxx

Buffer objects can now be used when setting the response body of any Foxx action. This allows Foxx actions to return binary data.

Requests with binary payload can be processed in Foxx applications by using the new method  res.rawBodyBuffer() . This will return the unparsed request body as a Buffer object.

There is now also the method  req.requestParts()  available in Foxx to retrieve the individual components of a multipart HTTP request. That can be used for example to process file uploads.

Additionally, the  res.send()  method has been added as a convenience method for returning strings, JSON objects or Buffers from a Foxx action. It provides some auto-detection based on its parameter value:

res.send("<p>some HTML</p>");  // returns an HTML string

res.send({ success: true });   // returns a JSON object

res.send(new Buffer("some binary data"));  // returns binary data

The convenience method  res.sendFile()  can now be used to return the contents of a file from a Foxx action. They file may contain binary data:

res.sendFile(applicationContext.foxxFilename("image.png"));

The filesystem methods  fs.write()  and  fs.readBuffer()  can be used to work with binary data, too:

fs.write()  will perform an auto-detection of its second parameter's value so it works with Buffer objects:

fs.write(filename, "some data");  // saves a string value in file

fs.write(filename, new Buffer("some binary data"));  // saves (binary) contents of a buffer

fs.readBuffer()  has been added as a method to read the contents of an arbitrary file into a Buffer object.

Web interface

Batch document removal and move functionality has been added to the web interface, making it easier to work with multiple documents at once. Additionally, basic JSON import and export tools have been added.

Command-line options added

The command-line option  --javascript.v8-contexts  was added to arangod to provide better control over the number of V8 contexts created in arangod.

Previously, the number of V8 contexts arangod created at startup was equal to the number of server threads (as specified by option  --server.threads ).

In some situations it may be more sensible to create different amounts of threads and V8 contexts. This is because each V8 contexts created will consume memory and requires CPU resources for periodic garbage collection. Contrary, server threads do not have such high memory or CPU footprint.

If the option  --javascript.v8-contexts  is not specified, the number of V8 contexts created at startup will remain equal to the number of server threads. Thus no change in configuration is required to keep the same behavior as in previous ArangoDB versions.

The command-line option  --log.use-local-time  was added to print dates and times in ArangoDB's log in the server-local timezone instead of UTC. If it is not set, the timezone will default to UTC.

The option  --backslash-escape  has been added to arangoimp. Specifying this option will use the backslash as the escape character for literal quotes when parsing CSV files. The escape character for literal quotes is still the double quote character.

Miscellaneous improvements

ArangoDB's built-in HTTP server now supports HTTP pipelining.

The ArangoShell tutorial from the arangodb.com website is now integrated into the ArangoDB shell.

Powerful Foxx Enhancements

With the new job queue feature you can run async jobs to communicate with external services, Foxx queries make writing complex AQL queries much easier and Foxx sessions will handle the authentication and session hassle for you.

Foxx Queries

Writing long AQL queries in JavaScript can quickly become unwieldy. As of 2.3 ArangoDB bundles the ArangoDB Query Builder module that provides a JavaScript API for writing complex AQL queries without string concatenation. All built-in functions that accept AQL strings now support query builder instances directly. Additionally Foxx provides a method  Foxx.createQuery  for creating parametrized queries that can return Foxx models or apply arbitrary transformations to the query results.

Foxx Sessions

The session functionality in Foxx has been completely rewritten. The old  activateAuthentication  API is still supported but may be deprecated in the future. The new  activateSessions  API supports cookies or configurable headers, provides optional JSON Web Token and cryptographic signing support and uses the new sessions Foxx app.

ArangoDB 2.3 provides Foxx apps for user management and salted hash-based authentication which can be replaced with or supplemented by alternative implementations. For an example app using both the built-in authentication and OAuth2 see the Foxx Sessions Example app.

Foxx Queues

Foxx now provides async workers via the Foxx Queues API. Jobs enqueued in a job queue will be executed asynchronously outside of the request/response cycle of Foxx controllers and can be used to communicate with external services or perform tasks that take a long time to complete or may require multiple attempts.

Jobs can be scheduled in advance or set to be executed immediately, the number of retry attempts, the retry delay as well as success and failure handlers can be defined for each job individually. Job types that integrate various external services for transactional e-mails, logging and user tracking can be found in the Foxx app registry.

Misc

The request and response objects in Foxx controllers now provide methods for reading and writing raw cookies and signed cookies.

Mounted Foxx apps will now be loaded when arangod starts rather than at the first database request. This may result in slightly slower start up times (but a faster response for the first request).

Features and Improvements

The following list shows in detail which features have been added or improved in ArangoDB 2.2. ArangoDB 2.2 also contains several bugfixes that are not listed here.

AQL improvements

Data modification AQL queries

Up to including version 2.1, AQL supported data retrieval operations only. Starting with ArangoDB version 2.2, AQL also supports the following data modification operations:

- INSERT: insert new documents into a collection

- UPDATE: partially update existing documents in a collection

- REPLACE: completely replace existing documents in a collection

- REMOVE: remove existing documents from a collection

Data-modification operations are normally combined with other AQL statements such as FOR loops and FILTER conditions to determine the set of documents to operate on. For example, the following query will find all documents in collection users that match a specific condition and set their status variable to inactive:

FOR u IN users

FILTER u.status == 'not active'

UPDATE u WITH { status: 'inactive' } IN users

The following query copies all documents from collection users into collection backup:

FOR u IN users

INSERT u IN backup

And this query removes documents from collection backup:

FOR doc IN backup

FILTER doc.lastModified < DATE_NOW() - 3600

REMOVE doc IN backup

For more information on data-modification queries, please refer to Data modification queries.

Updatable variables

Previously, the value of a variable assigned in an AQL query with the  LET  keyword was not updatable in an AQL query. This prevented statements like the following from being executable:

LET sum = 0

FOR v IN values

SORT v.year

LET sum = sum + v.value

RETURN { year: v.year, value: v.value, sum: sum }

Other AQL improvements

- added AQL TRANSLATE function

This function can be used to perform lookups from static objects, e.g.

LET countryNames = { US: "United States", UK: "United Kingdom", FR: "France" }

RETURN TRANSLATE("FR", countryNames)

LET lookup = { foo: "foo-replacement", bar: "bar-replacement", baz: "baz-replacement" }

RETURN TRANSLATE("foobar", lookup, "not contained!")

Write-ahead log

All write operations in an ArangoDB server will now be automatically logged in the server's write-ahead log. The write-ahead log is a set of append-only logfiles, and it is used in case of a crash recovery and for replication.

Data from the write-ahead log will eventually be moved into the journals or datafiles of collections, allowing the server to remove older write-ahead logfiles.

Cross-collection transactions in ArangoDB should benefit considerably by this change, as less writes than in previous versions are required to ensure the data of multiple collections are atomically and durably committed. All data-modifying operations inside transactions (insert, update, remove) will write their operations into the write-ahead log directly now. In previous versions, such operations were buffered until the commit or rollback occurred. Transactions with multiple operations should therefore require less physical memory than in previous versions of ArangoDB.

The data in the write-ahead log can also be used in the replication context. In previous versions of ArangoDB, replicating from a master required turning on a special replication logger on the master. The replication logger caused an extra write operation into the _replication system collection for each actual write operation. This extra write is now superfluous. Instead, slaves can read directly from the master's write-ahead log to get informed about most recent data changes. This removes the need to store data-modification operations in the _replication collection altogether.

For the configuration of the write-ahead log, please refer to Write-ahead log options.

The introduction of the write-ahead log also removes the need to configure and start the replication logger on a master. Though the replication logger object is still available in ArangoDB 2.2 to ensure API compatibility, starting, stopping, or configuring it will have no effect.

Performance improvements

- Removed sorting of attribute names when in collection shaper

In previous versions of ArangoDB, adding a document with previously not-used attribute names caused a full sort of all attribute names used in the collection. The sorting was done to ensure fast comparisons of attribute names in some rare edge cases, but it considerably slowed down inserts into collections with many different or even unique attribute names.

- Specialized primary index implementation to allow faster hash table rebuilding and reduce lookups in datafiles for the actual value of  _key . This also reduces the amount of random memory accesses for primary index inserts.

- Reclamation of index memory when deleting last document in collection

Deleting documents from a collection did not lead to index sizes being reduced. Instead, the index memory was kept allocated and re-used later when a collection was refilled with new documents. Now, index memory of primary indexes and hash indexes is reclaimed instantly when the last document in a collection is removed.

- Prevent buffering of long print results in arangosh's and arangod's print command

This change will emit buffered intermediate print results and discard the output buffer to quickly deliver print results to the user, and to prevent constructing very large buffers for large results.

Miscellaneous improvements

- Added  insert  method as an alias for  save . Documents can now be inserted into a collection using either method:

db.test.save({ foo: "bar" });

db.test.insert({ foo: "bar" });

- Cleanup of options for data-modification operations

Many of the data-modification operations had signatures with many optional bool parameters, e.g.:

db.test.update("foo", { bar: "baz" }, true, true, true)

db.test.replace("foo", { bar: "baz" }, true, true)

db.test.remove("foo", true, true)

db.test.save({ bar: "baz" }, true)

Such long parameter lists were unintuitive and hard to use when only one of the optional parameters should have been set.

To make the APIs more usable, the operations now understand the following alternative signature:

collection.update(key, update-document, options)

collection.replace(key, replacement-document, options)

collection.remove(key, options)

collection.save(document, options)

Examples:

db.test.update("foo", { bar: "baz" }, { overwrite: true, keepNull: true, waitForSync: true })

db.test.replace("foo", { bar: "baz" }, { overwrite: true, waitForSync: true })

db.test.remove("foo", { overwrite: true, waitForSync: true })

db.test.save({ bar: "baz" }, { waitForSync: true })

- Added  --overwrite  option to arangoimp

This allows removing all documents in a collection before importing into it using arangoimp.

- Honor startup option  --server.disable-statistics  when deciding whether or not to start periodic statistics collection jobs

Previously, the statistics collection jobs were started even if the server was started with the  --server.disable-statistics  flag being set to  true . Now if the option is set to  true , no statistics will be collected on the server.

- Disallow storing of JavaScript objects that contain JavaScript native objects of type  Date ,  Function ,  RegExp  or  External , e.g.

db.test.save({ foo: /bar/ });

db.test.save({ foo: new Date() });

This will now print

Error: <data> cannot be converted into JSON shape: could not shape document

Previously, objects of these types were silently converted into an empty object (i.e.  { } ) and no warning was issued.

To store such objects in a collection, explicitly convert them into strings like this:

db.test.save({ foo: String(/bar/) });

db.test.save({ foo: String(new Date()) });

Removed features

MRuby integration for arangod

ArangoDB had an experimental MRuby integration in some of the publish builds. This wasn't continuously developed, and so it has been removed in ArangoDB 2.2.

This change has led to the following startup options being superfluous:

- --ruby.gc-interval

- --ruby.action-directory

- --ruby.modules-path

- --ruby.startup-directory

Specifying these startup options will do nothing in ArangoDB 2.2, so using these options should be avoided from now on as they might be removed in a future version of ArangoDB.

Removed startup options

The following startup options have been removed in ArangoDB 2.2. Specifying them in the server's configuration file will not produce an error to make migration easier. Still, usage of these options should be avoided as they will not have any effect and might fully be removed in a future version of ArangoDB:

- --database.remove-on-drop

- --database.force-sync-properties

- --random.no-seed

- --ruby.gc-interval

- --ruby.action-directory

- --ruby.modules-path

- --ruby.startup-directory

- --server.disable-replication-logger

Multi Collection Graphs

ArangoDB is a multi model database with native graph support. In version 2.2 the features for graphs have been improved by integration of a new graph module. All graphs created with the old module are automatically migrated into the new module but can still be used by the old module.

New graph module

Up to including version 2.1, ArangoDB offered a module for graphs and graph operations. This module allowed you to use exactly one edge collection together with one vertex collection in a graph. With ArangoDB version 2.2 this graph module is deprecated and a new graph module is offered. This new module allows to combine an arbitrary number of vertex collections and edge collections in the same graph. For each edge collection a list of collections containing source vertices and a list of collections containing target vertices can be defined. If an edge is stored ArangoDB checks if this edge is valid in this collection. Furthermore if a vertex is removed from one of the collections all connected edges will be removed as well, giving the guarantee of no loose ends in the graphs. The layout of the graph can be modified at runtime by adding or removing collections and changing the definitions for edge collections. All operations on the graph level are transactional by default.

Graphs in AQL

Multi collection graphs have been added to AQL as well. Basic functionality (getting vertices, edges, neighbors) can be executed using the entire graph. Also more advanced features like shortest path calculations, characteristic factors of the graph or traversals have been integrated into AQL. For these functions all graphs created with the graph module can be used.

Features and Improvements

The following list shows in detail which features have been added or improved in ArangoDB 2.1. ArangoDB 2.1 also contains several bugfixes that are not listed here.

New Edges Index

The edges index (used to store connections between nodes in a graph) internally uses a new data structure. This data structure improves the performance when populating the edge index (i.e. when loading an edge collection). For large graphs loading can be 20 times faster than with ArangoDB 2.0.

Additionally, the new index fixes performance problems that occurred when many duplicate  _from  or  _to  values were contained in the index. Furthermore, the new index supports faster removal of edges.

Finally, when loading an existing collection and building the edges index for the collection, less memory re-allocations will be performed.

Overall, this should considerably speed up loading edge collections.

The new index type replaces the old edges index type automatically, without any changes being required by the end user.

The API of the new index is compatible with the API of the old index. Still it is possible that the new index returns edges in a different order than the old index. This is still considered to be compatible because the old index had never guaranteed any result order either.

AQL Improvements

AQL offers functionality to work with dates. Dates are no data types of their own in AQL (neither they are in JSON, which is often used as a format to ship data into and out of ArangoDB). Instead, dates in AQL are internally represented by either numbers (timestamps) or strings. The date functions in AQL provide mechanisms to convert from a numeric timestamp to a string representation and vice versa.

There are two date functions in AQL to create dates for further use:

- DATE_TIMESTAMP(date)  Creates a UTC timestamp value from  date

- DATE_TIMESTAMP(year, month, day, hour, minute, second, millisecond) : Same as before, but allows specifying the individual date components separately. All parameters after  day  are optional.

- DATE_ISO8601(date) : Returns an ISO8601 datetime string from  date . The datetime string will always use UTC time, indicated by the  Z  at its end.

- DATE_ISO8601(year, month, day, hour, minute, second, millisecond) : same as before, but allows specifying the individual date components separately. All parameters after  day  are optional.

These two above date functions accept the following input values:

- numeric timestamps, indicating the number of milliseconds elapsed since the UNIX epoch (i.e. January 1st 1970 00:00:00 UTC). An example timestamp value is  1399472349522 , which translates to  2014-05-07T14:19:09.522Z .

- datetime strings in formats  YYYY-MM-DDTHH:MM:SS.MMM ,  YYYY-MM-DD HH:MM:SS.MMM , or  YYYY-MM-DD . Milliseconds are always optional.

A timezone difference may optionally be added at the end of the string, with the hours and minutes that need to be added or subtracted to the datetime value. For example,  2014-05-07T14:19:09+01:00  can be used to specify a one hour offset, and  2014-05-07T14:19:09+07:30  can be specified for seven and half hours offset. Negative offsets are also possible. Alternatively to an offset, a  Z  can be used to indicate UTC / Zulu time.

An example value is  2014-05-07T14:19:09.522Z  meaning May 7th 2014, 14:19:09 and 522 milliseconds, UTC / Zulu time. Another example value without time component is  2014-05-07Z .

Please note that if no timezone offset is specified in a datestring, ArangoDB will assume UTC time automatically. This is done to ensure portability of queries across servers with different timezone settings, and because timestamps will always be UTC-based.

- individual date components as separate function arguments, in the following order:

- year

- month

- day

- hour

- minute

- second

- millisecond

All components following  day  are optional and can be omitted. Note that no timezone offsets can be specified when using separate date components, and UTC / Zulu time will be used.

The following calls to  DATE_TIMESTAMP  are equivalent and will all return  1399472349522 :

DATE_TIMESTAMP("2014-05-07T14:19:09.522")

DATE_TIMESTAMP("2014-05-07T14:19:09.522Z")

DATE_TIMESTAMP("2014-05-07 14:19:09.522")

DATE_TIMESTAMP("2014-05-07 14:19:09.522Z")

DATE_TIMESTAMP(2014, 5, 7, 14, 19, 9, 522)

DATE_TIMESTAMP(1399472349522)

The same is true for calls to  DATE_ISO8601  that also accepts variable input formats:

DATE_ISO8601("2014-05-07T14:19:09.522Z")

DATE_ISO8601("2014-05-07 14:19:09.522Z")

DATE_ISO8601(2014, 5, 7, 14, 19, 9, 522)

DATE_ISO8601(1399472349522)

The above functions are all equivalent and will return  "2014-05-07T14:19:09.522Z" .

The following date functions can be used with dates created by  DATE_TIMESTAMP  and  DATE_ISO8601 :

- DATE_DAYOFWEEK(date) : Returns the weekday number of  date . The return values have the following meanings:

- 0: Sunday

- 1: Monday

- 2: Tuesday

- 3: Wednesday

- 4: Thursday

- 5: Friday

- 6: Saturday

- DATE_YEAR(date) : Returns the year part of  date  as a number.

- DATE_MONTH(date) : Returns the month part of  date  as a number.

- DATE_DAY(date) : Returns the day part of  date  as a number.

- DATE_HOUR(date) : Returns the hour part of  date  as a number.

- DATE_MINUTE(date) : Returns the minute part of  date  as a number.

- DATE_SECOND(date) : Returns the seconds part of  date  as a number.

- DATE_MILLISECOND(date) : Returns the milliseconds part of  date  as a number.

The following other date functions are also available:

- DATE_NOW() : Returns the current time as a timestamp.

Note that this function is evaluated on every invocation and may return different values when invoked multiple times in the same query.

The following other AQL functions have been added in ArangoDB 2.1:

- FLATTEN : this function can turn an array of sub-arrays into a single flat array. All array elements in the original array will be expanded recursively up to a configurable depth. The expanded values will be added to the single result array.

Example:

FLATTEN([ 1, 2, [ 3, 4 ], 5, [ 6, 7 ], [ 8, [ 9, 10 ] ])

will expand the sub-arrays on the first level and produce:

[ 1, 2, 3, 4, 5, 6, 7, 8, [ 9, 10 ] ]

To fully flatten the array, the maximum depth can be specified (e.g. with a value of  2 ):

FLATTEN([ 1, 2, [ 3, 4 ], 5, [ 6, 7 ], [ 8, [ 9, 10 ] ], 2)

This will fully expand the sub-arrays and produce:

[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ]

- CURRENT_DATABASE : this function will return the name of the database the current query is executed in.

- CURRENT_USER : this function returns the name of the current user that is executing the query. If authorization is turned off or the query is executed outside of a request context, no user is present and the function will return  null .

Cluster Dump and Restore

The dump and restore tools, arangodump and arangorestore, can now be used to dump and restore collections in a cluster. Additionally, a collection dump from a standalone ArangoDB server can be imported into a cluster, and vice versa.

Web Interface Improvements

The web interface in version 2.1 has a more compact dashboard. It provides charts with time-series for incoming requests, HTTP transfer volume and some server resource usage figures.

Additionally it provides trend indicators (e.g. 15 min averages) and distribution charts (aka histogram) for some figures.

Foxx Improvements

To easily access a file inside the directory of a Foxx application from within Foxx, Foxx's  applicationContext  now provides the  foxxFilename()  function. It can be used to assemble the full filename of a file inside the application's directory. The  applicationContext  can be accessed as global variable from any module within a Foxx application.

The filename can be used inside Foxx actions or setup / teardown scripts, e.g. to populate a Foxx application's collection with data.

The  require  function now also prefers local modules when used from inside a Foxx application. This allows putting modules inside the Foxx application directory and requiring them easily. It also allows using application-specific versions of libraries that are bundled with ArangoDB (such as underscore.js).

Windows Installer

The Windows installer shipped with ArangoDB now supports installation of ArangoDB for the current user or all users, with the required privileges. It also supports the installation of ArangoDB as a service.

Fixes for 32 bit systems

Several issues have been fixed that occurred only when using ArangoDB on a 32 bits operating system, specifically:

- a crash in a third party component used to manage cluster data

- a third party library that failed to initialize on 32 bit Windows, making arangod and arangosh crash immediately.

- overflows of values used for nanosecond-precision timeouts: these overflows have led to invalid values being passed to socket operations, making them fail and re-try too often

Updated drivers

Several drivers for ArangoDB have been checked for compatibility with 2.1. The current list of drivers with compatibility notes can be found online here.

C++11 usage

We have moved several files from C to C++, allowing more code reuse and reducing the need for shipping data between the two. We have also decided to require C++11 support for ArangoDB, which allows us to use some of the simplifications, features and guarantees that this standard has in stock.

That also means a compiler with C++11 support is required to build ArangoDB from source. For instance GNU CC of at least version 4.8.

Miscellaneous Improvements

- Cancelable asynchronous jobs: several potentially long-running jobs can now be canceled via an explicit cancel operation. This allows stopping long-running queries, traversals or scripts without shutting down the complete ArangoDB process. Job cancelation is provided for asynchronously executed jobs as is described in @ref HttpJobCancel.

- Server-side periodic task management: an ArangoDB server now provides functionality to register and unregister periodic tasks. Tasks are user-defined JavaScript actions that can be run periodically and automatically, independent of any HTTP requests.

The following task management functions are provided:

- require("org/arangodb/tasks").register(): registers a periodic task

- require("org/arangodb/tasks").unregister(): unregisters and removes a periodic task

- require("org/arangodb/tasks").get(): retrieves a specific tasks or all existing tasks

An example task (to be executed every 15 seconds) can be registered like this:

var tasks = require("org/arangodb/tasks");

tasks.register({

name: "this is an example task with parameters",

period: 15,

command: function (params) {

var greeting = params.greeting;

var data = JSON.stringify(params.data);

require('console').log('%s from parameter task: %s', greeting, data);

},

params: { greeting: "hi", data: "how are you?" }

});

Please refer to the section @ref Tasks for more details.

- The  figures  method of a collection now returns data about the collection's index memory consumption. The returned value  indexes.size  will contain the total amount of memory acquired by all indexes of the collection. This figure can be used to assess the memory impact of indexes.

- Capitalized HTTP response headers: from version 2.1, ArangoDB will return capitalized HTTP headers by default, e.g.  Content-Length  instead of  content-length . Though the HTTP specification states that headers field name are case-insensitive, several older client tools rely on a specific case in HTTP response headers. This changes make ArangoDB a bit more compatible with those.

- Simplified usage of  db._createStatement() : to easily run an AQL query, the method  db._createStatement  now allows passing the AQL query as a string. Previously it required the user to pass an object with a  query  attribute (which then contained the query string).

ArangoDB now supports both versions:

db._createStatement(queryString);

db._createStatement({ query: queryString });

First Steps in ArangoDB

For installation instructions, please refer to the Installation Manual.

As you know from the introduction ArangoDB is a multi-model open-source Database. You can see the Key features below or look directly at the programs in the ArangoDB package.

Key features include:

- Schema-free schemata: Lets you combine the space efficiency of MySQL with the performance power of NoSQL

- Application server: Use ArangoDB as an application server and fuse your application and database together for maximal throughput

- JavaScript for all: No language zoo, you can use one language from your browser to your back-end

- Flexible data modeling: Model your data as combinations of key-value pairs, documents or graphs - perfect for social relations

- Free index choice: Use the correct index for your problem, may it be a skip list or a fulltext search

- Configurable durability: Let the application decide if it needs more durability or more performance

- No-nonsense storage: ArangoDB uses all of the power of modern storage hardware, like SSD and large caches

- Powerful query language (AQL) to retrieve and modify data

- Transactions: Run queries on multiple documents or collections with optional transactional consistency and isolation

- Replication: Set up the database in a master-slave configuration

- It is open source (Apache License 2.0)

For more in-depth information:

- Read more on the Design Goals of ArangoDB

- Watch the video: Martin Schönert, architect of ArangoDB, gives an introduction of what the ArangoDB project is about

- Or give it a try

ArangoDB programs

The ArangoDB package comes with the following programs:

- arangod: The ArangoDB database daemon. This server program is intended to run as a daemon process and to serve the various clients connection to the server via TCP / HTTP. See Details about the ArangoDB Server

- arangosh: The ArangoDB shell. A client that implements a read-eval-print loop (REPL) and provides functions to access and administrate the ArangoDB server. See Details about the ArangoDB Shell.

- arangoimp: A bulk importer for the ArangoDB server. See Details about Arangoimp.

- arangodump: A tool to create backups of an ArangoDB database. See Details about Arangodump.

- arangorestore: A tool to reload data from a backup into an ArangoDB database. See Details about Arangorestore

- arango-dfdb: A datafile debugger for ArangoDB. It is intended to be used primarily during development of ArangoDB

Getting familiar with ArangoDB

First of all download and install the corresponding RPM or Debian package or use homebrew on the MacOS X. See the installation manual for more details. In case you just want to experiment with ArangoDB you can use the online demo without installing ArangoDB locally.

For Linux

- Visit the official ArangoDB install page and download the correct package for your Linux distribution

- Install the package using your favorite package manager

- Start up the database server, normally this is done by executing /etc/init.d/arangod start. The exact command depends on your Linux distribution

For MacOS X

- Execute brew install arangodb

- And start the server using /usr/local/sbin/arangod &

For Microsoft Windows

- Visit the official ArangoDB install page and download the installer for Windows

- Start up the database server

After these steps there should be a running instance of arangod - the ArangoDB database server.

unix> ps auxw | fgrep arangod

arangodb 14536 0.1 0.6 5307264 23464 s002 S 1:21pm 0:00.18 /usr/local/sbin/arangod

If there is no such process, check the log file /var/log/arangodb/arangod.log for errors. If you see a log message like

2012-12-03T11:35:29Z [12882] ERROR Database directory version (1) is lower than server version (1.2).

2012-12-03T11:35:29Z [12882] ERROR It seems like you have upgraded the ArangoDB binary. If this is what you wanted to do, please restart with the --upgrade option to upgrade the data in the database directory.

2012-12-03T11:35:29Z [12882] FATAL Database version check failed. Please start the server with the --upgrade option

make sure to start the server once with the --upgrade option.

Details about the ArangoDB Server

The ArangoDB database server has two modes of operation: As a server, where it will answer to client requests and as an emergency console, in which you can access the database directly. The latter - as the name suggests - should only be used in case of an emergency, for example, a corrupted collection. Using the emergency console allows you to issue all commands normally available in actions and transactions. When starting the server in emergency console mode, the server cannot handle any client requests.

You should never start more than one server using the same database directory, independent from the mode of operation. Normally ArangoDB will prevent you from doing this by placing a lockfile in the database directory and not allowing a second ArangoDB instance to use the same database directory if a lockfile is already present.

The following command starts the ArangoDB database in server mode. You will be able to access the server using HTTP requests on port 8529. Look here for a list of frequently used options – see here for a complete list.

unix> /usr/local/sbin/arangod /tmp/vocbase

20ZZ-XX-YYT12:37:08Z [8145] INFO using built-in JavaScript startup files

20ZZ-XX-YYT12:37:08Z [8145] INFO ArangoDB (version 1.x.y) is ready for business

20ZZ-XX-YYT12:37:08Z [8145] INFO Have Fun!

After starting the server, point your favorite browser to:

http://localhost:8529/

to access the administration front-end.

Linux

To start the server at system boot time you should use one of the pre-rolled packages that will install the necessary start / stop scripts for ArangoDB. You can use the start script as follows:

unix> /etc/init.d/arangod start

To stop the server you can use the following command:

unix> /etc/init.d/arangod stop

You may require root privileges to execute these commands.

If you compiled ArangoDB from source and did not use any installation package – or using non-default locations and/or multiple ArangoDB instances on the same host – you may want to start the server process manually. You can do so by invoking the arangod binary from the command line as shown before. To stop the database server gracefully, you can either press CTRL-C or by send the SIGINT signal to the server process. On many systems this can be achieved with the following command:

unix> kill -2 `pidof arangod`

Frequently Used Options

The following command-line options are frequently used. For a full list of options see the documentation.

database-directory

Uses the "database-directory" as base directory. There is an alternative version available for use in configuration files, see configuration documentation.

--help

-h

Prints a list of the most common options available and then exists. In order to see all options use  --help-all .

--log level

Allows the user to choose the level of information which is logged by the server. The "level" is specified as a string and can be one of the following values: fatal, error, warning, info, debug or trace. For more information see here.

--server.disable-authentication

Setting value to true will turn off authentication on the server side so all clients can execute any action without authorization and privilege checks.

The default value is false.

--server.keep-alive-timeout

Allows to specify the timeout for HTTP keep-alive connections. The timeout value must be specified in seconds. Idle keep-alive connections will be closed by the server automatically when the timeout is reached. A keep-alive-timeout value 0 will disable the keep alive feature entirely.

--server.endpoint endpoint

Specifies an endpoint for HTTP requests by clients. Endpoints have the following pattern:

- tcp://ipv4-address:port - TCP/IP endpoint, using IPv4

- tcp://[ipv6-address]:port - TCP/IP endpoint, using IPv6

- ssl://ipv4-address:port - TCP/IP endpoint, using IPv4, SSL encryption

- ssl://[ipv6-address]:port - TCP/IP endpoint, using IPv6, SSL encryption

- unix:///path/to/socket - Unix domain socket endpoint

If a TCP/IP endpoint is specified without a port number, then the default port (8529) will be used. If multiple endpoints need to be used, the option can be repeated multiple times.

Examples

unix> ./arangod --server.endpoint tcp://127.0.0.1:8529

--server.endpoint ssl://127.0.0.1:8530

--server.keyfile server.pem /tmp/vocbase

2012-07-26T07:07:47Z [8161] INFO using SSL protocol version 'TLSv1'

2012-07-26T07:07:48Z [8161] INFO using endpoint 'ssl://127.0.0.1:8530' for http ssl requests

2012-07-26T07:07:48Z [8161] INFO using endpoint 'tcp://127.0.0.1:8529' for http tcp requests

2012-07-26T07:07:49Z [8161] INFO ArangoDB (version 1.1.alpha) is ready for business

2012-07-26T07:07:49Z [8161] INFO Have Fun!

Note: If you are using SSL-encrypted endpoints, you must also supply the path to a server certificate using the --server.keyfile option.

Endpoints can also be changed at runtime. Please refer to HTTP Interface for Endpoints for more details.  --daemon

Runs the server as a "daemon" (as a background process).

Troubleshooting

If the ArangoDB server does not start or if you cannot connect to it using arangosh or other clients, you can try to find the problem cause by executing the following steps. If the server starts up without problems you can skip this section.

- Check the server log file: If the server has written a log file you should check it because it might contain relevant error context information.

- Check the configuration: The server looks for a configuration file named arangod.conf on startup. The contents of this file will be used as a base configuration that can optionally be overridden with command-line configuration parameters. You should check the config file for the most relevant parameters such as:

- server.endpoint: What IP address and port to bind to

- log parameters: If and where to log

- database.directory: Path the database files are stored in

If the configuration reveals that something is not configured right the config file should be adjusted and the server be restarted.

- Start the server manually and check its output: Starting the server might fail even before logging is activated so the server will not produce log output. This can happen if the server is configured to write the logs to a file that the server has no permissions on. In this case the server cannot log an error to the specified log file but will write a startup error on stderr instead. Starting the server manually will also allow you to override specific configuration options, e.g. to turn on/off file or screen logging etc.

- Check the TCP port: If the server starts up but does not accept any incoming connections this might be due to firewall configuration between the server and any client(s). The server by default will listen on TCP port 8529. Please make sure this port is actually accessible by other clients if you plan to use ArangoDB in a network setup.

When using hostnames in the configuration or when connecting, please make sure the hostname is actually resolvable. Resolving hostnames might invoke DNS, which can be a source of errors on its own.

It is generally good advice to not use DNS when specifying the endpoints and connection addresses. Using IP addresses instead will rule out DNS as a source of errors. Another alternative is to use a hostname specified in the local /etc/hosts file, which will then bypass DNS.

- Test if curl can connect: Once the server is started, you can quickly verify if it responds to requests at all. This check allows you to determine whether connection errors are client-specific or not. If at least one client can connect, it is likely that connection problems of other clients are not due to ArangoDB's configuration but due to client or in-between network configurations.

You can test connectivity using a simple command such as:

curl --dump - -X GET http://127.0.0.1:8529/_api/version && echo

This should return a response with an HTTP 200 status code when the server is running. If it does it also means the server is generally accepting connections. Alternative tools to check connectivity are lynx or ab.

ArangoDB Shell Introduction

The ArangoDB shell (arangosh) is a command-line tool that can be used for administration of ArangoDB, including running ad-hoc queries.

The arangosh binary is shipped with ArangoDB. It offers a JavaScript shell environment providing access to the ArangoDB server. Arangosh can be invoked like this:

unix> arangosh

By default arangosh will try to connect to an ArangoDB server running on server localhost on port 8529. It will use the username root and an empty password by default. Additionally it will connect to the default database (_system). All these defaults can be changed using the following command-line options:

- --server.database : name of the database to connect to

- --server.endpoint : endpoint to connect to

- --server.username : database username

- --server.password : password to use when connecting

- --server.disable-authentication : disable password prompt and authentication

For example, to connect to an ArangoDB server on IP 192.168.173.13 on port 8530 with the user foo and using the database test, use:

unix> arangosh  \

--server.endpoint tcp://192.168.173.13:8530  \

--server.username foo  \

--server.database test  \

--server.disable-authentication false

arangosh will then display a password prompt and try to connect to the server after the password was entered.

To change the current database after the connection has been made, you can use the  db._useDatabase()  command in arangosh:

arangosh> db._createDatabase("myapp");

true

arangosh> db._useDatabase("myapp");

true

arangosh> db._useDatabase("_system");

true

arangosh> db._dropDatabase("myapp");

true

To get a list of available commands, arangosh provides a help() function. Calling it will display helpful information.

arangosh also provides auto-completion. Additional information on available commands and methods is thus provided by typing the first few letters of a variable and then pressing the tab key. It is recommend to try this with entering db. (without pressing return) and then pressing tab.

By the way, arangosh provides the db object by default, and this object can be used for switching to a different database and managing collections inside the current database.

For a list of available methods for the db object, type

arangosh> db._help();

show execution results

you can paste multiple lines into arangosh, given the first line ends with an opening brace:

arangosh> for (var i = 0; i < 10; i ++) {

........>         require("internal").print("Hello world " + i + "!

");

........> }

show execution results

To load your own JavaScript code into the current JavaScript interpreter context, use the load command:

require("internal").load("/tmp/test.js")     // <- Linux / MacOS

require("internal").load("c:\\tmp\\test.js") // <- Windows

Exiting arangosh can be done using the key combination  <CTRL> + D  or by typing  quit<CR>

ArangoDB Shell Output

By default, the ArangoDB shell uses a pretty printer when JSON documents are printed. This ensures documents are printed in a human-readable way:

arangosh> db._create("five")

arangosh> for (i = 0; i < 5; i++) db.five.save({value:i})

arangosh> db.five.toArray()

show execution results

While the pretty-printer produces nice looking results, it will need a lot of screen space for each document. Sometimes, a more dense output might be better. In this case, the pretty printer can be turned off using the command stop_pretty_print().

To turn on pretty printing again, use the start_pretty_print() command.

ArangoDB Shell Configuration

arangosh will look for a user-defined startup script named .arangosh.rc in the user's home directory on startup. The home directory will likely be  /home/<username>/  on Unix/Linux, and is determined on Windows by peeking into the environment variables  %HOMEDRIVE%  and  %HOMEPATH% .

If the file .arangosh.rc is present in the home directory, arangosh will execute the contents of this file inside the global scope.

You can use this to define your own extra variables and functions that you need often. For example, you could put the following into the .arangosh.rc file in your home directory:

// "var" keyword omitted intentionally...

// otherwise "timed" would not survive the scope of this script

global.timed = function (cb) {

var internal = require("internal");

var start = internal.time();

cb();

internal.print("execution took: ", internal.time() - start);

};

This will make a function named timed available in arangosh in the global scope.

You can now start arangosh and invoke the function like this:

timed(function () {

for (var i = 0; i < 1000; ++i) {

db.test.save({ value: i });

}

});

Please keep in mind that, if present, the .arangosh.rc file needs to contain valid JavaScript code. If you want any variables in the global scope to survive you need to omit the var keyword for them. Otherwise the variables will only be visible inside the script itself, but not outside.

Details about the ArangoDB Shell

After the server has been started, you can use the ArangoDB shell (arangosh) to administrate the server. Without any arguments, the ArangoDB shell will try to contact the server on port 8529 on the localhost. For more information see the ArangoDB Shell documentation. You might need to set additional options (endpoint, username and password) when connecting:

unix> ./arangosh --server.endpoint tcp://127.0.0.1:8529 --server.username root

The shell will print its own version number and – if successfully connected to a server – the version number of the ArangoDB server.

Command-Line Options

Use  --help  to get a list of command-line options:

unix> ./arangosh --help

STANDARD options:

--audit-log <string>          audit log file to save commands and results to

--configuration <string>      read configuration file

--help                        help message

--max-upload-size <uint64>    maximum size of import chunks (in bytes) (default: 500000)

--no-auto-complete            disable auto completion

--no-colors                   deactivate color support

--pager <string>              output pager (default: "less -X -R -F -L")

--pretty-print                pretty print values

--quiet                       no banner

--temp-path <string>          path for temporary files (default: "/tmp/arangodb")

--use-pager                   use pager

JAVASCRIPT options:

--javascript.check <string>                syntax check code JavaScript code from file

--javascript.execute <string>              execute JavaScript code from file

--javascript.execute-string <string>       execute JavaScript code from string

--javascript.startup-directory <string>    startup paths containing the JavaScript files

--javascript.unit-tests <string>           do not start as shell, run unit tests instead

--jslint <string>                          do not start as shell, run jslint instead

LOGGING options:

--log.level <string>    log level (default: "info")

CLIENT options:

--server.connect-timeout <double>         connect timeout in seconds (default: 3)

--server.disable-authentication <bool>    disable authentication (default: false)

--server.endpoint <string>                endpoint to connect to, use 'none' to start without a server (default: "tcp://127.0.0.1:8529")

--server.password <string>                password to use when connecting (leave empty for prompt)

--server.request-timeout <double>         request timeout in seconds (default: 300)

--server.username <string>                username to use when connecting (default: "root")

Database Wrappers

The db-Object is available in arangosh as well as on arangod i.e. if you're using Foxx. While its Interface is persistant bewteen the arangosh and the arangod implementations, its underpinning are not. The arangod implementation is JavaScript wrappers around ArangoDBs native C++ implementation, the arangosh implementation wraps HTTP-Accesses to ArangoDBs RESTfull API.

So while this code may produce similar results when executed in arangosh and arangod, the cpu-usage and time required will be really different:

for (i = 0; i < 100000; i++) {

db.test.save({ name: { first: "Jan" }, count: i});

}

Since the arangosh version will be doing around 100k HTTP-Requests, and the arangod version will directly write to the database.

Exploring Collections and Documents

ArangoDB is a database that serves documents to clients.

- A document contains zero or more attributes, each one of these attributes has a value. A value can either be an atomic type, i. e. integer, strings, boolean, an array or an embedded document. Documents are normally represented as JSON objects

- Documents are grouped into collections. A collection contains zero or more documents

- Queries are used to filter documents based on certain criteria. Queries can be as simple as a "query by example" or as complex as "joins" using many collections or graph structures

- Cursors are used to iterate over the result of a query

- Indexes are used to speed up searches. There are various types of indexes such as hash indexes and geo indexes

If you are familiar with RDBMS then it is safe to compare collections to tables and documents to rows. However, bringing structure to the "rows" has many advantages - as you will see later.

Starting the JavaScript shell

The easiest way to connect to the database is the JavaScript shell arangosh. You can either start it from the command-line or as an embedded version in the browser. Using the command-line tool has the advantage that you can use autocompletion.

unix> arangosh --server.password ""

_

__ _ _ __ __ _ _ __   __ _  ___  ___| |__

/ _` | '__/ _` | '_ \ / _` |/ _ \/ __| '_ \

| (_| | | | (_| | | | | (_| | (_) \__ \ | | |

\__,_|_|  \__,_|_| |_|\__, |\___/|___/_| |_|

|___/

Welcome to arangosh 2.x.y. Copyright (c) 2012 triAGENS GmbH.

Using Google V8 4.1.0.27 JavaScript engine.

Using READLINE 6.1.

Connected to Arango DB 127.0.0.1:8529 Version 2.2.0

arangosh> help

------------------------------------- Help -------------------------------------

Predefined objects:

arango:                               ArangoConnection

db:                                   ArangoDatabase

fm:                                   FoxxManager

Example:

> db._collections();                   list all collections

> db._create(<name>)                   create a new collection

> db._drop(<name>)                     drop a collection

> db.<name>.toArray()                  list all documents

> id = db.<name>.save({ ... })         save a document

> db.<name>.remove(<_id>)              delete a document

> db.<name>.document(<_id>)            retrieve a document

> db.<name>.replace(<_id>, {...})      overwrite a document

> db.<name>.update(<_id>, {...})       partially update a document

> db.<name>.exists(<_id>)              check if document exists

> db._query(<query>).toArray()         execute an AQL query

> db._useDatabase(<name>)              switch database

> db._createDatabase(<name>)           create a new database

> db._listDatabases()                  list existing databases

> help                                 show help pages

> exit

arangosh>

This gives you a prompt where you can issue JavaScript commands.

The standard setup does not require a password. Depending on your setup you might need to specify the endpoint, username and password in order to run the shell on your system. You can use the options  --server.endpoint ,  --server.username  and  --server.password  for this.

unix> arangosh --server.endpoint tcp://127.0.0.1:8529 --server.username root

A default configuration is normally installed under /etc/arangodb/arangosh.conf. It contains a default endpoint and an empty password.

Querying for Documents

All documents are stored in collections. All collections are stored in a database. The database object is accessible via the variable db.

Creating a collection is simple. You can use the _create method of the db variable.

arangosh> db._create("example");

[ArangoCollection 450359528, "example" (type document, status loaded)]

After the collection has been created you can easily access it using the path db.example. The collection currently shows as loaded, meaning that it's loaded into memory. If you restart the server and access the collection again it will now show as unloaded. You can also manually unload a collection.

arangosh> db.example.unload();

arangosh> db.example;

[ArangoCollection 450359528, "example" (type document, status unloaded)]

Whenever you use a collection ArangoDB will automatically load it into memory for you.

In order to create new documents in a collection use the save operation.

arangosh> db.example.save({ Hello : "World" });

arangosh> db.example.save({ "name" : "John Doe", "age" : 29 });

arangosh> db.example.save({ "name" : "Jane Smith", "age" : 31 });

show execution results

Just storing documents would be no fun. We now want to select some of the stored documents again. In order to select all elements of a collection, one can use the toArray method:

arangosh> db.example.toArray()

show execution results

Now we want to look for a person with a given name. We can use byExample for this. The method returns an array of documents matching a given example.

arangosh> db.example.byExample({ name: "Jane Smith" }).toArray()

show execution results

While the byExample works very well for simple queries where you combine the conditions with an  and . The syntax above becomes messy for joins and or conditions. Therefore ArangoDB also supports a full-blown query language, AQL. To run an AQL query, use the db._query method:.

arangosh> db._query('FOR user IN example FILTER user.name == "Jane Smith" RETURN user').toArray()

show execution results

Searching for all persons with an age above 30:

arangosh> db._query('FOR user IN example FILTER user.age > 30 RETURN user').toArray()

show execution results

John was put in there by mistake – so let's delete him again; We fetch the  _id  using byExample:

arangosh> db.example.remove(db.example.byExample({ name: "John Doe" }).toArray()[0]._id)

arangosh> db.example.toArray()

show execution results

You can learn all about the query language Aql. Note that _query is a short-cut for _createStatement and execute. We will come back to these functions when we talk about cursors.

ArangoDB's Front-End

The ArangoDB server has a graphical front-end, which allows you to inspect the current state of the server from within your browser. You can use the front-end using the following URL:

http://localhost:8529/

The front-end allows you to browse through the collections and documents. If you need to administrate the database, please use the ArangoDB shell described in the next section.

Accessing the Web Interface

ArangoDB comes with a built-in web interface for administration. The web interface can be accessed via the URL

http://localhost:8529

assuming you are using the standard port and no user routings. If you have any service installed, the home page might point to that service instead. In this case use

http://localhost:8529/_admin/aardvark/index.html

(note: aardvark is the web interface's internal name).

If no database name is specified in the URL, you will in most cases get routed to the web interface for the _system database. To access the web interface for any other ArangoDB database, put the database name into the request URI path as follows:

http://localhost:8529/_db/mydb/

The above will load the web interface for the database mydb.

To restrict access to the web interface, use ArangoDB's authentication feature.

Select Functionality provided by the Web Interface

The following sections provide a very brief overview of some features offered in the web interface. Please note that this is not a complete list of features.

Dashboard Tab

The Dashboard tab provides statistics which are polled regularly from the ArangoDB server.

Collections Tab

The Collections tab shows an overview of the loaded and unloaded collections present in ArangoDB. System collections (i.e. collections whose names start with an underscore) are not shown by default.

The list of collections can be restricted using the search bar or by using the filtering at the top. The filter can also be used to show or hide system collections.

Clicking on a collection will show the documents contained in it. Clicking the small icon on a collection's badge will bring up a dialog that allows loading/unloading, renaming and deleting the collection.

Please note that you should not change or delete system collections.

In the list of documents of a collection, you can click on the Add document line to add a new document to the collection. The document will be created instantly, with a system-defined key. The key and all other attributes of the document can be adjusted in the following view.

Services Tab

The Services tab provides a list of installed Foxx services ( also called applications). The view is divided into lists of installed and services that are available for installation.

Please note that ArangoDB's web interface (aardvark) is a Foxx service itself. Please also note that installed services will be listed in both the installed and the available section. This is intentional because each service can be installed multiple times using different mount points.

Graphs Tab

The Graphs tab provides a viewer facility for graph data stored in ArangoDB. It allows browsing ArangoDB graphs stored in the _graphs system collection or a graph consisting of an arbitrary vertex and edge collection.

Please note that the graph viewer requires client-side SVG and that you need a browser capable of rendering that. Especially Internet Explorer browsers older than version 9 are likely to not support this.

AQL Editor Tab

The AQL Editor tab allows to execute ad-hoc AQL queries.

Type in a query in the bottom box and execute it by pressing the Submit button. The query result will be shown in the box at the top. The editor provides a few example queries that can be used as templates.

There is also the option to add own frequently used queries here. Note that own queries will be stored in the browser's local storage and the web interface has no control over when the browser's local storage is cleared.

Tools Tab

The Tools tab contains a JavaScript shell that can be used to run commands on the ArangoDB server, a log viewer and a link to the description of ArangoDB's REST API.

The JS Shell menu item provides access to a JavaScript shell that connects to the database server.

Any valid JavaScript code can be executed inside the shell. The code will be executed inside your browser. To contact the ArangoDB server you can use the db object, for example as follows:

JSH> db._create("mycollection");

JSH> db.mycollection.save({ _key: "test", value: "something" });

You can use the Logs menu item allows browsing the most recent log entries provided by the ArangoDB database server.

Note that the server only keeps a limited number of log entries. For real log analyses write the logs to disk using syslog or a similar mechanism. ArangoDB provides several startup options for this.

The Logs menu item will only be shown for the  _system  database, and is disabled for any other databases.

The API menu item provides an overview of ArangoDB's built-in HTTP REST API, with documentation and examples. It should be consulted when there is doubt about API URLs, parameters etc.

Handling Databases

This is an introduction to managing databases in ArangoDB from within JavaScript.

When you have an established connection to ArangoDB, the current database can be changed explicitly using the db._useDatabase() method. This will switch to the specified database (provided it exists and the user can connect to it). From this point on, any following action in the same shell or connection will use the specified database, unless otherwise specified.

Note: If the database is changed, client drivers need to store the current database name on their side, too. This is because connections in ArangoDB do not contain any state information. All state information is contained in the HTTP request/response data.

To connect to a specific database after arangosh has started use the command described above. It is also possible to specify a database name when invoking arangosh. For this purpose, use the command-line parameter --server.database, e.g.

> arangosh --server.database test

Please note that commands, actions, scripts or AQL queries should never access multiple databases, even if they exist. The only intended and supported way in ArangoDB is to use one database at a time for a command, an action, a script or a query. Operations started in one database must not switch the database later and continue operating in another.

Working with Databases

Database Methods

The following methods are available to manage databases via JavaScript. Please note that several of these methods can be used from the _system database only.

Name

db._name()

Returns the name of the current database as a string.

Examples

arangosh> require("internal").db._name();

_system

ID

db._id()

Returns the id of the current database as a string.

Examples

arangosh> require("internal").db._id();

127208

Path

db._path()

Returns the filesystem path of the current database as a string.

Examples

arangosh> require("internal").db._path();

/tmp/vocdir.26839/databases/database-127208

isSystem

db._isSystem()

Returns whether the currently used database is the _system database. The system database has some special privileges and properties, for example, database management operations such as create or drop can only be executed from within this database. Additionally, the _system database itself cannot be dropped.

Use Database

db._useDatabase(name)

Changes the current database to the database specified by name. Note that the database specified by name must already exist.

Changing the database might be disallowed in some contexts, for example server-side actions (including Foxx).

When performing this command from arangosh, the current credentials (username and password) will be re-used. These credentials might not be valid to connect to the database specified by name. Additionally, the database only be accessed from certain endpoints only. In this case, switching the database might not work, and the connection / session should be closed and restarted with different username and password credentials and/or endpoint data.

List Databases

db._listDatabases()

Returns the list of all databases. This method can only be used from within the _system database.

Create Database

db._createDatabase(name, options, users)

Creates a new database with the name specified by name. There are restrictions for database names (see DatabaseNames).

Note that even if the database is created successfully, there will be no change into the current database to the new database. Changing the current database must explicitly be requested by using the db._useDatabase method.

The options attribute currently has no meaning and is reserved for future use.

The optional users attribute can be used to create initial users for the new database. If specified, it must be a list of user objects. Each user object can contain the following attributes:

- username: the user name as a string. This attribute is mandatory.

- passwd: the user password as a string. If not specified, then it defaults to the empty string.

- active: a boolean flag indicating whether the user account should be active or not. The default value is true.

- extra: an optional JSON object with extra user information. The data contained in extra will be stored for the user but not be interpreted further by ArangoDB.

If no initial users are specified, a default user root will be created with an empty string password. This ensures that the new database will be accessible via HTTP after it is created.

You can create users in a database if no initial user is specified. Switch into the new database (username and password must be identical to the current session) and add or modify users with the following commands.

require("org/arangodb/users").save(username, password, true);

require("org/arangodb/users").update(username, password, true);

require("org/arangodb/users").remove(username);

Alternatively, you can specify user data directly. For example:

db._createDatabase("newDB", [], [{ username: "newUser", passwd: "123456", active: true}])

Those methods can only be used from within the _system database.

Drop Database

db._dropDatabase(name)

Drops the database specified by name. The database specified by name must exist.

Note: Dropping databases is only possible from within the _system database. The _system database itself cannot be dropped.

Databases are dropped asynchronously, and will be physically removed if all clients have disconnected and references have been garbage-collected.

Notes about Databases

Please keep in mind that each database contains its own system collections, which need to be set up when a database is created. This will make the creation of a database take a while.

Replication is configured on a per-database level, meaning that any replication logging or applying for a new database must be configured explicitly after a new database has been created.

Foxx applications are also available only in the context of the database they have been installed in. A new database will only provide access to the system applications shipped with ArangoDB (that is the web interface at the moment) and no other Foxx applications until they are explicitly installed for the particular database.

JavaScript Interface to Collections

This is an introduction to ArangoDB's interface for collections and how to handle collections from the JavaScript shell arangosh. For other languages see the corresponding language API.

The most important call is the call to create a new collection.

Address of a Collection

All collections in ArangoDB have a unique identifier and a unique name. ArangoDB internally uses the collection's unique identifier to look up collections. This identifier, however, is managed by ArangoDB and the user has no control over it. In order to allow users to use their own names, each collection also has a unique name which is specified by the user. To access a collection from the user perspective, the collection name should be used, i.e.:

Collection

db._collection(collection-name)

A collection is created by a "db._create" call.

For example: Assume that the collection identifier is 7254820 and the name is demo, then the collection can be accessed as:

db._collection("demo")

If no collection with such a name exists, then null is returned.

There is a short-cut that can be used for non-system collections:

Collection name

db.collection-name

This call will either return the collection named db.collection-name or create a new one with that name and a set of default properties.

Note: Creating a collection on the fly using db.collection-name is not recommend and does not work in arangosh. To create a new collection, please use

Create

db._create(collection-name)

This call will create a new collection called collection-name. This method is a database method and is documented in detail at Database Methods

Collection Methods

Drop

collection.drop()

Drops a collection and all its indexes.

Examples

arangosh> col = db.example;

[ArangoCollection 1080815848, "example" (type document, status loaded)]

arangosh> col.drop();

arangosh> col;

[ArangoCollection 1080815848, "example" (type document, status deleted)]

Truncate

collection.truncate()

Truncates a collection, removing all documents but keeping all its indexes.

Examples

Truncates a collection:

arangosh> col = db.example;

arangosh> col.save({ "Hello" : "World" });

arangosh> col.count();

arangosh> col.truncate();

arangosh> col.count();

show execution results

Properties

collection.properties()

Returns an object containing all collection properties.

- waitForSync: If true creating a document will only return after the data was synced to disk.

- journalSize : The size of the journal in bytes.

- isVolatile: If true then the collection data will be kept in memory only and ArangoDB will not write or sync the data to disk.

- keyOptions (optional) additional options for key generation. This is a JSON array containing the following attributes (note: some of the attributes are optional):

- type: the type of the key generator used for the collection.

- allowUserKeys: if set to true, then it is allowed to supply own key values in the _key attribute of a document. If set to false, then the key generator will solely be responsible for generating keys and supplying own key values in the _key attribute of documents is considered an error.

- increment: increment value for autoincrement key generator. Not used for other key generator types.

- offset: initial offset value for autoincrement key generator. Not used for other key generator types.

- indexBuckets: number of buckets into which indexes using a hash table are split. The default is 16 and this number has to be a power of 2 and less than or equal to 1024.

For very large collections one should increase this to avoid long pauses when the hash table has to be initially built or resized, since buckets are resized individually and can be initially built in parallel. For example, 64 might be a sensible value for a collection with 100 000 000 documents. Currently, only the edge index respects this value, but other index types might follow in future ArangoDB versions. Changes (see below) are applied when the collection is loaded the next time.

In a cluster setup, the result will also contain the following attributes:

- numberOfShards: the number of shards of the collection.

- shardKeys: contains the names of document attributes that are used to determine the target shard for documents.

collection.properties(properties)

Changes the collection properties. properties must be a object with one or more of the following attribute(s):

- waitForSync: If true creating a document will only return after the data was synced to disk.

- journalSize : The size of the journal in bytes.

- indexBuckets : See above, changes are only applied when the collection is loaded the next time.

Note: it is not possible to change the journal size after the journal or datafile has been created. Changing this parameter will only effect newly created journals. Also note that you cannot lower the journal size to less then size of the largest document already stored in the collection.

Note: some other collection properties, such as type, isVolatile, or keyOptions cannot be changed once the collection is created.

Examples

Read all properties

arangosh> db.example.properties();

show execution results

Change a property

arangosh> db.example.properties({ waitForSync : true });

show execution results

Figures

collection.figures()

Returns an object containing statistics about the collection. Note : Retrieving the figures will always load the collection into memory.

- alive.count: The number of currently active documents in all datafiles and journals of the collection. Documents that are contained in the write-ahead log only are not reported in this figure.

- alive.size: The total size in bytes used by all active documents of the collection. Documents that are contained in the write-ahead log only are not reported in this figure.

- dead.count: The number of dead documents. This includes document versions that have been deleted or replaced by a newer version. Documents deleted or replaced that are contained in the write-ahead log only are not reported in this figure.

- dead.size: The total size in bytes used by all dead documents.

- dead.deletion: The total number of deletion markers. Deletion markers only contained in the write-ahead log are not reporting in this figure.

- datafiles.count: The number of datafiles.

- datafiles.fileSize: The total filesize of datafiles (in bytes).

- journals.count: The number of journal files.

- journals.fileSize: The total filesize of the journal files (in bytes).

- compactors.count: The number of compactor files.

- compactors.fileSize: The total filesize of the compactor files (in bytes).

- shapefiles.count: The number of shape files. This value is deprecated and kept for compatibility reasons only. The value will always be 0 since ArangoDB 2.0 and higher.

- shapefiles.fileSize: The total filesize of the shape files. This value is deprecated and kept for compatibility reasons only. The value will always be 0 in ArangoDB 2.0 and higher.

- shapes.count: The total number of shapes used in the collection. This includes shapes that are not in use anymore. Shapes that are contained in the write-ahead log only are not reported in this figure.

- shapes.size: The total size of all shapes (in bytes). This includes shapes that are not in use anymore. Shapes that are contained in the write-ahead log only are not reported in this figure.

- attributes.count: The total number of attributes used in the collection. Note: the value includes data of attributes that are not in use anymore. Attributes that are contained in the write-ahead log only are not reported in this figure.

- attributes.size: The total size of the attribute data (in bytes). Note: the value includes data of attributes that are not in use anymore. Attributes that are contained in the write-ahead log only are not reported in this figure.

- indexes.count: The total number of indexes defined for the collection, including the pre-defined indexes (e.g. primary index).

- indexes.size: The total memory allocated for indexes in bytes.

- maxTick: The tick of the last marker that was stored in a journal of the collection. This might be 0 if the collection does not yet have a journal.

- uncollectedLogfileEntries: The number of markers in the write-ahead log for this collection that have not been transferred to journals or datafiles.

- documentReferences: The number of references to documents in datafiles that JavaScript code currently holds. This information can be used for debugging compaction and unload issues.

- waitingFor: An optional string value that contains information about which object type is at the head of the collection's cleanup queue. This information can be used for debugging compaction and unload issues.

- compactionStatus.time: The point in time the compaction for the collection was last executed. This information can be used for debugging compaction issues.

- compactionStatus.message: The action that was performed when the compaction was last run for the collection. This information can be used for debugging compaction issues.

Note: collection data that are stored in the write-ahead log only are not reported in the results. When the write-ahead log is collected, documents might be added to journals and datafiles of the collection, which may modify the figures of the collection. Also note that  waitingFor  and  compactionStatus  may be empty when called on a coordinator in a cluster.

Additionally, the filesizes of collection and index parameter JSON files are not reported. These files should normally have a size of a few bytes each. Please also note that the fileSize values are reported in bytes and reflect the logical file sizes. Some filesystems may use optimisations (e.g. sparse files) so that the actual physical file size is somewhat different. Directories and sub-directories may also require space in the file system, but this space is not reported in the fileSize results.

That means that the figures reported do not reflect the actual disk usage of the collection with 100% accuracy. The actual disk usage of a collection is normally slightly higher than the sum of the reported fileSize values. Still the sum of the fileSize values can still be used as a lower bound approximation of the disk usage.

Examples

arangosh> db.demo.figures()

show execution results

Load

collection.load()

Loads a collection into memory.

Examples

arangosh> col = db.example;

[ArangoCollection 1091760360, "example" (type document, status loaded)]

arangosh> col.load();

arangosh> col;

[ArangoCollection 1091760360, "example" (type document, status loaded)]

Reserve

collection.reserve( number)

Sends a resize hint to the indexes in the collection. The resize hint allows indexes to reserve space for additional documents (specified by number) in one go.

The reserve hint can be sent before a mass insertion into the collection is started. It allows indexes to allocate the required memory at once and avoids re-allocations and possible re-locations.

Not all indexes implement the reserve function at the moment. The indexes that don't implement it will simply ignore the request. returns the revision id of a collection

Revision

collection.revision()

Returns the revision id of the collection

The revision id is updated when the document data is modified, either by inserting, deleting, updating or replacing documents in it.

The revision id of a collection can be used by clients to check whether data in a collection has changed or if it is still unmodified since a previous fetch of the revision id.

The revision id returned is a string value. Clients should treat this value as an opaque string, and only use it for equality/non-equality comparisons.

Checksum

collection.checksum(withRevisions, withData)

The checksum operation calculates a CRC32 checksum of the keys contained in collection collection.

If the optional argument withRevisions is set to true, then the revision ids of the documents are also included in the checksumming.

If the optional argument withData is set to true, then the actual document data is also checksummed. Including the document data in checksumming will make the calculation slower, but is more accurate.

Note: this method is not available in a cluster.

Unload

collection.unload()

Starts unloading a collection from memory. Note that unloading is deferred until all query have finished.

Examples

arangosh> col = db.example;

[ArangoCollection 522252520, "example" (type document, status loaded)]

arangosh> col.unload();

arangosh> col;

[ArangoCollection 522252520, "example" (type document, status unloaded)]

Rename

collection.rename(new-name)

Renames a collection using the new-name. The new-name must not already be used for a different collection. new-name must also be a valid collection name. For more information on valid collection names please refer to the naming conventions.

If renaming fails for any reason, an error is thrown. If renaming the collection succeeds, then the collection is also renamed in all graph definitions inside the  _graphs  collection in the current database.

Note: this method is not available in a cluster.

Examples

arangosh> c = db.example;

[ArangoCollection 1098641640, "example" (type document, status loaded)]

arangosh> c.rename("better-example");

arangosh> c;

[ArangoCollection 1098641640, "better-example" (type document, status loaded)]

Rotate

collection.rotate()

Rotates the current journal of a collection. This operation makes the current journal of the collection a read-only datafile so it may become a candidate for garbage collection. If there is currently no journal available for the collection, the operation will fail with an error.

Note: this method is not available in a cluster.

Database Methods

Collection

db._collection(collection-name)

Returns the collection with the given name or null if no such collection exists.

db._collection(collection-identifier)

Returns the collection with the given identifier or null if no such collection exists. Accessing collections by identifier is discouraged for end users. End users should access collections using the collection name.

Examples

Get a collection by name:

arangosh> db._collection("demo");

[ArangoCollection 16511208, "demo" (type document, status loaded)]

Get a collection by id:

arangosh> db._collection(123456);

[ArangoCollection 123456, "demo" (type document, status loaded)]

Unknown collection:

arangosh> db._collection("unknown");

null

Create

db._create(collection-name)

Creates a new document collection named collection-name. If the collection name already exists or if the name format is invalid, an error is thrown. For more information on valid collection names please refer to the naming conventions.

db._create(collection-name, properties)

properties must be an object with the following attributes:

- waitForSync (optional, default false): If true creating a document will only return after the data was synced to disk.

- journalSize (optional, default is a configuration parameter: The maximal size of a journal or datafile. Note that this also limits the maximal size of a single object. Must be at least 1MB.

- isSystem (optional, default is false): If true, create a system collection. In this case collection-name should start with an underscore. End users should normally create non-system collections only. API implementors may be required to create system collections in very special occasions, but normally a regular collection will do.

- isVolatile (optional, default is false): If true then the collection data is kept in-memory only and not made persistent. Unloading the collection will cause the collection data to be discarded. Stopping or re-starting the server will also cause full loss of data in the collection. Setting this option will make the resulting collection be slightly faster than regular collections because ArangoDB does not enforce any synchronization to disk and does not calculate any CRC checksums for datafiles (as there are no datafiles).

- keyOptions (optional): additional options for key generation. If specified, then keyOptions should be a JSON array containing the following attributes (note: some of them are optional):

- type: specifies the type of the key generator. The currently available generators are traditional and autoincrement.

- allowUserKeys: if set to true, then it is allowed to supply own key values in the _key attribute of a document. If set to false, then the key generator will solely be responsible for generating keys and supplying own key values in the _key attribute of documents is considered an error.

- increment: increment value for autoincrement key generator. Not used for other key generator types.

- offset: initial offset value for autoincrement key generator. Not used for other key generator types.

- numberOfShards (optional, default is 1): in a cluster, this value determines the number of shards to create for the collection. In a single server setup, this option is meaningless.

- shardKeys (optional, default is [ "_key" ]): in a cluster, this attribute determines which document attributes are used to determine the target shard for documents. Documents are sent to shards based on the values they have in their shard key attributes. The values of all shard key attributes in a document are hashed, and the hash value is used to determine the target shard. Note that values of shard key attributes cannot be changed once set. This option is meaningless in a single server setup.

When choosing the shard keys, one must be aware of the following rules and limitations: In a sharded collection with more than one shard it is not possible to set up a unique constraint on an attribute that is not the one and only shard key given in shardKeys. This is because enforcing a unique constraint would otherwise make a global index necessary or need extensive communication for every single write operation. Furthermore, if _key is not the one and only shard key, then it is not possible to set the _key attribute when inserting a document, provided the collection has more than one shard. Again, this is because the database has to enforce the unique constraint on the _key attribute and this can only be done efficiently if this is the only shard key by delegating to the individual shards.

db._create(collection-name, properties, type)

Specifies the optional type of the collection, it can either be document or edge. On default it is document. Instead of giving a type you can also use db._createEdgeCollection or db._createDocumentCollection.

Examples

With defaults:

arangosh> c = db._create("users");

arangosh> c.properties();

show execution results

With properties:

arangosh> c = db._create("users", { waitForSync : true, journalSize : 1024 * 1204 });

arangosh> c.properties();

show execution results

With a key generator:

arangosh> db._create("users", { keyOptions: { type: "autoincrement", offset: 10, increment: 5 } });

arangosh> db.users.save({ name: "user 1" });

arangosh> db.users.save({ name: "user 2" });

arangosh> db.users.save({ name: "user 3" });

show execution results

With a special key option:

arangosh> db._create("users", { keyOptions: { allowUserKeys: false } });

arangosh> db.users.save({ name: "user 1" });

arangosh> db.users.save({ name: "user 2", _key: "myuser" });

arangosh> db.users.save({ name: "user 3" });

show execution results

db._createEdgeCollection(collection-name)

Creates a new edge collection named collection-name. If the collection name already exists an error is thrown. The default value for waitForSync is false.

db._createEdgeCollection(collection-name, properties)

properties must be an object with the following attributes:

- waitForSync (optional, default false): If true creating a document will only return after the data was synced to disk.

- journalSize (optional, default is "configuration parameter"): The maximal size of a journal or datafile. Note that this also limits the maximal size of a single object and must be at least 1MB.

db._createDocumentCollection(collection-name)

Creates a new document collection named collection-name. If the document name already exists and error is thrown.

All Collections

db._collections()

Returns all collections of the given database.

Examples

arangosh> db._collections();

show execution results

Collection Name

db.collection-name

Returns the collection with the given collection-name. If no such collection exists, create a collection named collection-name with the default properties.

Examples

arangosh> db.example;

[ArangoCollection 1076097256, "example" (type document, status loaded)]

Drop

db._drop(collection)

Drops a collection and all its indexes.

db._drop(collection-identifier)

Drops a collection identified by collection-identifier and all its indexes. No error is thrown if there is no such collection.

db._drop(collection-name)

Drops a collection named collection-name and all its indexes. No error is thrown if there is no such collection.

Examples

Drops a collection:

arangosh> col = db.example;

[ArangoCollection 1078456552, "example" (type document, status loaded)]

arangosh> db._drop(col);

arangosh> col;

[ArangoCollection 1078456552, "example" (type document, status loaded)]

Drops a collection identified by name:

arangosh> col = db.example;

[ArangoCollection 1078653160, "example" (type document, status loaded)]

arangosh> db._drop("example");

arangosh> col;

[ArangoCollection 1078653160, "example" (type document, status deleted)]

Truncate

db._truncate(collection)

Truncates a collection, removing all documents but keeping all its indexes.

db._truncate(collection-identifier)

Truncates a collection identified by collection-identified. No error is thrown if there is no such collection.

db._truncate(collection-name)

Truncates a collection named collection-name. No error is thrown if there is no such collection.

Examples

Truncates a collection:

arangosh> col = db.example;

arangosh> col.save({ "Hello" : "World" });

arangosh> col.count();

arangosh> db._truncate(col);

arangosh> col.count();

show execution results

Truncates a collection identified by name:

arangosh> col = db.example;

arangosh> col.save({ "Hello" : "World" });

arangosh> col.count();

arangosh> db._truncate("example");

arangosh> col.count();

show execution results

Documents, Identifiers, Handles

This is an introduction to ArangoDB's interface for documents to and how handle documents from the JavaScript shell arangosh. For other languages see the corresponding language API.

Documents in ArangoDB are JSON objects. These objects can be nested (to any depth) and may contain lists. Each document is uniquely identified by its document handle.

For example:

{

"_id" : "myusers/3456789",

"_key" : "3456789",

"_rev" : "14253647",

"firstName" : "John",

"lastName" : "Doe",

"address" : {

"city" : "Gotham",

"street" : "Road To Nowhere 1"

},

"hobbies" : [

{name: "swimming", howFavorite: 10},

{name: "biking", howFavorite: 6},

{name: "programming", howFavorite: 4}

]

}

All documents contain special attributes: the document handle in _id, the document's unique key in _key and and the ETag aka document revision in _rev. The value of the _key attribute can be specified by the user when creating a document. _id and _key values are immutable once the document has been created. The _rev value is maintained by ArangoDB autonomously.

A document handle uniquely identifies a document in the database. It is a string and consists of the collection's name and the document key (_key attribute) separated by /.

As ArangoDB supports MVCC, documents can exist in more than one revision. The document revision is the MVCC token used to identify a particular revision of a document. It is a string value currently containing an integer number and is unique within the list of document revisions for a single document. Document revisions can be used to conditionally update, replace or delete documents in the database. In order to find a particular revision of a document, you need the document handle and the document revision.

ArangoDB currently uses 64bit unsigned integer values to maintain document revisions internally. When returning document revisions to clients, ArangoDB will put them into a string to ensure the revision id is not clipped by clients that do not support big integers. Clients should treat the revision id returned by ArangoDB as an opaque string when they store or use it locally. This will allow ArangoDB to change the format of revision ids later if this should be required. Clients can use revisions ids to perform simple equality/non-equality comparisons (e.g. to check whether a document has changed or not), but they should not use revision ids to perform greater/less than comparisons with them to check if a document revision is older than one another, even if this might work for some cases.

Note: Revision ids have been returned as integers up to including ArangoDB 1.1

Document Etag: The document revision enclosed in double quotes. The revision is returned by several HTTP API methods in the Etag HTTP header.

Coming from a relational background - how do browse vectors translate into document queries?

In traditional SQL you may either fetch all columns of a table row by row using  SELECT * FROM table , or select a subset of the columns. The list of table columns to fetch is commonly called column list or browse vector:  SELECT columnA, columnB, columnZ FROM table

Since documents aren't 2 dimensional, and neither you want only to be able to return 2 dimensional lists, requirements to AQL are a little bit more complex. The semantics how it does this leans mostly on the syntax used in JavaScript to work with structured data.

Composing the return documents

The AQL  RETURN  statement returns one item per document it is handed. You can Return the whole document, or just parts of it:

The full example:

RETURN oneDocument

-> {

"_id" : "myusers/3456789",

"_key" : "3456789",

"_rev" : "14253647",

"firstName" : "John",

"lastName" : "Doe",

"address" : {

"city" : "Gotham",

"street" : "Road To Nowhere 1"

},

"hobbies" : [

{name: "swimming", howFavorite: 10},

{name: "biking", howFavorite: 6},

{name: "programming", howFavorite: 4}

]

}

Only the hobbies sub-structure from the example:

RETURN oneDocument.hobbies

-> [

{name: "swimming", howFavorite: 10},

{name: "biking", howFavorite: 6},

{name: "programming", howFavorite: 4}

]

The hobbies and the address:

RETURN { hobbies: oneDocument.hobbies, address: oneDocument.address }

-> {

hobbies: [

{name: "swimming", howFavorite: 10},

{name: "biking", howFavorite: 6},

{name: "programming", howFavorite: 4}

],

address: {

"city" : "Gotham",

"street" : "Road To Nowhere 1"

},

}

The first hoby:

RETURN oneDocument.hobbies[0].name

->"swimming"

A list of the hobbie strings:

RETURN { hobbies: oneDocument.hobbies[*].name }

-> {hobbies: ["swimming", "biking", "porgramming"] }

More complex array and object manipulations can be done using AQL functions

Address and ETag of a Document

All documents in ArangoDB have a document handle. This handle uniquely defines a document and is managed by ArangoDB. The interface allows you to access the documents of a collection as:

db.collection.document("document-handle")

For example: Assume that the document handle, which is stored in the _id field of the document, is demo/362549 and the document lives in a collection named demo, then that document can be accessed as:

db.demo.document("demo/362549736")

Because the document handle is unique within the database, you can leave out the collection and use the shortcut:

db._document("demo/362549736")

Each document also has a document revision or ETag which is returned in the _rev field when requesting a document. The document's key is returned in the _key attribute.

Collection Methods

All

collection.all()

Fetches all documents from a collection and returns a cursor. You can use toArray, next, or hasNext to access the result. The result can be limited using the skip and limit operator.

Examples

Use toArray to get all documents at once:

arangosh> db.five.save({ name : "one" });

arangosh> db.five.save({ name : "two" });

arangosh> db.five.save({ name : "three" });

arangosh> db.five.save({ name : "four" });

arangosh> db.five.save({ name : "five" });

arangosh> db.five.all().toArray();

show execution results

Use limit to restrict the documents:

arangosh> db.five.save({ name : "one" });

arangosh> db.five.save({ name : "two" });

arangosh> db.five.save({ name : "three" });

arangosh> db.five.save({ name : "four" });

arangosh> db.five.save({ name : "five" });

arangosh> db.five.all().limit(2).toArray();

show execution results

Query by example

collection.byExample(example)

Fetches all documents from a collection that match the specified example and returns a cursor.

You can use toArray, next, or hasNext to access the result. The result can be limited using the skip and limit operator.

An attribute name of the form a.b is interpreted as attribute path, not as attribute. If you use

{ a : { c : 1 } }

as example, then you will find all documents, such that the attribute a contains a document of the form {c : 1 }. For example the document

{ a : { c : 1 }, b : 1 }

will match, but the document

{ a : { c : 1, b : 1 } }

will not.

However, if you use

{ a.c : 1 },

then you will find all documents, which contain a sub-document in a that has an attribute c of value 1. Both the following documents

{ a : { c : 1 }, b : 1 } and

{ a : { c : 1, b : 1 } }

will match.

collection.byExample(path1, value1, ...)

As alternative you can supply an array of paths and values.

Examples

Use toArray to get all documents at once:

arangosh> db.users.save({ name: "Gerhard" });

arangosh> db.users.save({ name: "Helmut" });

arangosh> db.users.save({ name: "Angela" });

arangosh> db.users.all().toArray();

arangosh> db.users.byExample({ "_id" : "users/20" }).toArray();

arangosh> db.users.byExample({ "name" : "Gerhard" }).toArray();

arangosh> db.users.byExample({ "name" : "Helmut", "_id" : "users/15" }).toArray();

show execution results

Use next to loop over all documents:

arangosh> db.users.save({ name: "Gerhard" });

arangosh> db.users.save({ name: "Helmut" });

arangosh> db.users.save({ name: "Angela" });

arangosh> var a = db.users.byExample( {"name" : "Angela" } );

arangosh> while (a.hasNext()) print(a.next());

show execution results

First Example

collection.firstExample(example)

Returns the first document of a collection that matches the specified example. If no such document exists, null will be returned. The example has to be specified as paths and values. See byExample for details.

collection.firstExample(path1, value1, ...)

As alternative you can supply an array of paths and values.

Examples

arangosh> db.users.firstExample("name", "Angela");

show execution results

Range

collection.range(attribute, left, right)

Returns all documents from a collection such that the attribute is greater or equal than left and strictly less than right.

You can use toArray, next, or hasNext to access the result. The result can be limited using the skip and limit operator.

An attribute name of the form a.b is interpreted as attribute path, not as attribute.

For range queries it is required that a skiplist index is present for the queried attribute. If no skiplist index is present on the attribute, an error will be thrown.

Note: the range simple query function is deprecated as of ArangoDB 2.6. The function may be removed in future versions of ArangoDB. The preferred way for retrieving documents from a collection within a specific range is to use an AQL query as follows:

FOR doc IN @@collection

FILTER doc.value >= @left && doc.value < @right

LIMIT @skip, @limit

RETURN doc

Examples

Use toArray to get all documents at once:

arangosh> db.old.ensureIndex({ type: "skiplist", fields: [ "age" ] });

arangosh> db.old.save({ age: 15 });

arangosh> db.old.save({ age: 25 });

arangosh> db.old.save({ age: 30 });

arangosh> db.old.range("age", 10, 30).toArray();

show execution results

Closed range

collection.closedRange(attribute, left, right)

Returns all documents of a collection such that the attribute is greater or equal than left and less or equal than right.

You can use toArray, next, or hasNext to access the result. The result can be limited using the skip and limit operator.

An attribute name of the form a.b is interpreted as attribute path, not as attribute.

Note: the closedRange simple query function is deprecated as of ArangoDB 2.6. The function may be removed in future versions of ArangoDB. The preferred way for retrieving documents from a collection within a specific range is to use an AQL query as follows:

FOR doc IN @@collection

FILTER doc.value >= @left && doc.value <= @right

LIMIT @skip, @limit

RETURN doc

Examples

Use toArray to get all documents at once:

arangosh> db.old.ensureIndex({ type: "skiplist", fields: [ "age" ] });

arangosh> db.old.save({ age: 15 });

arangosh> db.old.save({ age: 25 });

arangosh> db.old.save({ age: 30 });

arangosh> db.old.closedRange("age", 10, 30).toArray();

show execution results

Any

collection.any()

Returns a random document from the collection or null if none exists.

Count

collection.count()

Returns the number of living documents in the collection.

Examples

arangosh> db.users.count();

0

toArray

collection.toArray()

Converts the collection into an array of documents. Never use this call in a production environment.

Document

collection.document(document)

The document method finds a document given its identifier or a document object containing the _id or _key attribute. The method returns the document if it can be found.

An error is thrown if _rev is specified but the document found has a different revision already. An error is also thrown if no document exists with the given _id or _key value.

Please note that if the method is executed on the arangod server (e.g. from inside a Foxx application), an immutable document object will be returned for performance reasons. It is not possible to change attributes of this immutable object. To update or patch the returned document, it needs to be cloned/copied into a regular JavaScript object first. This is not necessary if the document method is called from out of arangosh or from any other client.

collection.document(document-handle)

As before. Instead of document a document-handle can be passed as first argument.

Examples

Returns the document for a document-handle:

arangosh> db.example.document("example/2873916");

show execution results

An error is raised if the document is unknown:

arangosh> db.example.document("example/4472917");

[ArangoError 1202: document not found]

An error is raised if the handle is invalid:

arangosh> db.example.document("");

[ArangoError 1205: illegal document handle]

Exists

collection.exists(document)

The exists method determines whether a document exists given its identifier. Instead of returning the found document or an error, this method will return either true or false. It can thus be used for easy existence checks.

The document method finds a document given its identifier. It returns the document. Note that the returned document contains two pseudo-attributes, namely _id and _rev. _id contains the document-handle and _rev the revision of the document.

No error will be thrown if the sought document or collection does not exist. Still this method will throw an error if used improperly, e.g. when called with a non-document handle, a non-document, or when a cross-collection request is performed.

collection.exists(document-handle)

As before. Instead of document a document-handle can be passed as first argument.

Lookup By Keys

collection.documents(keys)

Looks up the documents in the specified collection using the array of keys provided. All documents for which a matching key was specified in the keys array and that exist in the collection will be returned. Keys for which no document can be found in the underlying collection are ignored, and no exception will be thrown for them.

Examples

arangosh> keys = [ ];

arangosh> for (var i = 0; i < 10; ++i) {

........>   db.example.insert({ _key: "test" + i, value: i });

........>   keys.push("test" + i);

........> }

arangosh> db.example.documents(keys);

show execution results

Insert

collection.insert(data)

Creates a new document in the collection from the given data. The data must be an object.

The method returns a document with the attributes _id and _rev. The attribute _id contains the document handle of the newly created document, the attribute _rev contains the document revision.

collection.insert(data, waitForSync)

Creates a new document in the collection from the given data as above. The optional waitForSync parameter can be used to force synchronization of the document creation operation to disk even in case that the waitForSync flag had been disabled for the entire collection. Thus, the waitForSync parameter can be used to force synchronization of just specific operations. To use this, set the waitForSync parameter to true. If the waitForSync parameter is not specified or set to false, then the collection's default waitForSync behavior is applied. The waitForSync parameter cannot be used to disable synchronization for collections that have a default waitForSync value of true.

Note: since ArangoDB 2.2, insert is an alias for save.

Examples

arangosh> db.example.insert({ Hello : "World" });

arangosh> db.example.insert({ Hello : "World" }, true);

show execution results

Replace

collection.replace(document, data)

Replaces an existing document. The document must be a document in the current collection. This document is then replaced with the data given as second argument.

The method returns a document with the attributes _id, _rev and {_oldRev. The attribute _id contains the document handle of the updated document, the attribute _rev contains the document revision of the updated document, the attribute _oldRev contains the revision of the old (now replaced) document.

If there is a conflict, i. e. if the revision of the document does not match the revision in the collection, then an error is thrown.

collection.replace(document, data, true)  or  collection.replace(document, data, overwrite: true)

As before, but in case of a conflict, the conflict is ignored and the old document is overwritten.

collection.replace(document, data, true, waitForSync)  or  collection.replace(document, data, overwrite: true, waitForSync: true or false)

The optional waitForSync parameter can be used to force synchronization of the document replacement operation to disk even in case that the waitForSync flag had been disabled for the entire collection. Thus, the waitForSync parameter can be used to force synchronization of just specific operations. To use this, set the waitForSync parameter to true. If the waitForSync parameter is not specified or set to false, then the collection's default waitForSync behavior is applied. The waitForSync parameter cannot be used to disable synchronization for collections that have a default waitForSync value of true.

collection.replace(document-handle, data)

As before. Instead of document a document-handle can be passed as first argument.

Examples

Create and update a document:

arangosh> a1 = db.example.insert({ a : 1 });

arangosh> a2 = db.example.replace(a1, { a : 2 });

arangosh> a3 = db.example.replace(a1, { a : 3 });

show execution results

Use a document handle:

arangosh> a1 = db.example.insert({ a : 1 });

arangosh> a2 = db.example.replace("example/3903044", { a : 2 });

show execution results

Update

collection.update(document, data, overwrite, keepNull, waitForSync)  or  collection.update(document, data, overwrite: true or false, keepNull: true or false, waitForSync: true or false)

Updates an existing document. The document must be a document in the current collection. This document is then patched with the data given as second argument. The optional overwrite parameter can be used to control the behavior in case of version conflicts (see below). The optional keepNull parameter can be used to modify the behavior when handling null values. Normally, null values are stored in the database. By setting the keepNull parameter to false, this behavior can be changed so that all attributes in data with null values will be removed from the target document.

The optional waitForSync parameter can be used to force synchronization of the document update operation to disk even in case that the waitForSync flag had been disabled for the entire collection. Thus, the waitForSync parameter can be used to force synchronization of just specific operations. To use this, set the waitForSync parameter to true. If the waitForSync parameter is not specified or set to false, then the collection's default waitForSync behavior is applied. The waitForSync parameter cannot be used to disable synchronization for collections that have a default waitForSync value of true.

The method returns a document with the attributes _id, _rev and _oldRev. The attribute _id contains the document handle of the updated document, the attribute _rev contains the document revision of the updated document, the attribute _oldRev contains the revision of the old (now replaced) document.

If there is a conflict, i. e. if the revision of the document does not match the revision in the collection, then an error is thrown.

collection.update(document, data, true)

As before, but in case of a conflict, the conflict is ignored and the old document is overwritten.

collection.update(document-handle, data)`

As before. Instead of document a document-handle can be passed as first argument.

Examples

Create and update a document:

arangosh> a1 = db.example.insert({"a" : 1});

arangosh> a2 = db.example.update(a1, {"b" : 2, "c" : 3});

arangosh> a3 = db.example.update(a1, {"d" : 4});

arangosh> a4 = db.example.update(a2, {"e" : 5, "f" : 6 });

arangosh> db.example.document(a4);

arangosh> a5 = db.example.update(a4, {"a" : 1, c : 9, e : 42 });

arangosh> db.example.document(a5);

show execution results

Use a document handle:

arangosh> a1 = db.example.insert({"a" : 1});

arangosh> a2 = db.example.update("example/18612115", { "x" : 1, "y" : 2 });

show execution results

Use the keepNull parameter to remove attributes with null values:

arangosh> db.example.insert({"a" : 1});

arangosh> db.example.update("example/19988371", { "b" : null, "c" : null, "d" : 3 });

arangosh> db.example.document("example/19988371");

arangosh> db.example.update("example/19988371", { "a" : null }, false, false);

arangosh> db.example.document("example/19988371");

arangosh> db.example.update("example/19988371", { "b" : null, "c": null, "d" : null }, false, false);

arangosh> db.example.document("example/19988371");

show execution results

Patching array values:

arangosh> db.example.insert({"a" : { "one" : 1, "two" : 2, "three" : 3 }, "b" : { }});

arangosh> db.example.update("example/20774803", {"a" : { "four" : 4 }, "b" : { "b1" : 1 }});

arangosh> db.example.document("example/20774803");

arangosh> db.example.update("example/20774803", { "a" : { "one" : null }, "b" : null }, false, false);

arangosh> db.example.document("example/20774803");

show execution results

Remove

collection.remove(document)

Removes a document. If there is revision mismatch, then an error is thrown.

collection.remove(document, true)

Removes a document. If there is revision mismatch, then mismatch is ignored and document is deleted. The function returns true if the document existed and was deleted. It returns false, if the document was already deleted.

collection.remove(document, true, waitForSync)

The optional waitForSync parameter can be used to force synchronization of the document deletion operation to disk even in case that the waitForSync flag had been disabled for the entire collection. Thus, the waitForSync parameter can be used to force synchronization of just specific operations. To use this, set the waitForSync parameter to true. If the waitForSync parameter is not specified or set to false, then the collection's default waitForSync behavior is applied. The waitForSync parameter cannot be used to disable synchronization for collections that have a default waitForSync value of true.

collection.remove(document-handle, data)

As before. Instead of document a document-handle can be passed as first argument.

Examples

Remove a document:

arangosh> a1 = db.example.insert({ a : 1 });

arangosh> db.example.document(a1);

arangosh> db.example.remove(a1);

arangosh> db.example.document(a1);

show execution results

Remove a document with a conflict:

arangosh> a1 = db.example.insert({ a : 1 });

arangosh> a2 = db.example.replace(a1, { a : 2 });

arangosh> db.example.remove(a1);

arangosh> db.example.remove(a1, true);

arangosh> db.example.document(a1);

show execution results

Remove By Keys

collection.removeByKeys(keys)

Looks up the documents in the specified collection using the array of keys provided, and removes all documents from the collection whose keys are contained in the keys array. Keys for which no document can be found in the underlying collection are ignored, and no exception will be thrown for them.

The method will return an object containing the number of removed documents in the removed sub-attribute, and the number of not-removed/ignored documents in the ignored sub-attribute.

Examples

arangosh> keys = [ ];

arangosh> for (var i = 0; i < 10; ++i) {

........>   db.example.insert({ _key: "test" + i, value: i });

........>   keys.push("test" + i);

........> }

arangosh> db.example.removeByKeys(keys);

show execution results

Remove By Example

collection.removeByExample(example)

Removes all documents matching an example.

collection.removeByExample(document, waitForSync)

The optional waitForSync parameter can be used to force synchronization of the document deletion operation to disk even in case that the waitForSync flag had been disabled for the entire collection. Thus, the waitForSync parameter can be used to force synchronization of just specific operations. To use this, set the waitForSync parameter to true. If the waitForSync parameter is not specified or set to false, then the collection's default waitForSync behavior is applied. The waitForSync parameter cannot be used to disable synchronization for collections that have a default waitForSync value of true.

collection.removeByExample(document, waitForSync, limit)

The optional limit parameter can be used to restrict the number of removals to the specified value. If limit is specified but less than the number of documents in the collection, it is undefined which documents are removed.

Examples

arangosh> db.example.removeByExample( {Hello : "world"} );

1

Replace By Example

collection.replaceByExample(example, newValue)

Replaces all documents matching an example with a new document body. The entire document body of each document matching the example will be replaced with newValue. The document meta-attributes such as _id, _key, _from, _to will not be replaced.

collection.replaceByExample(document, newValue, waitForSync)

The optional waitForSync parameter can be used to force synchronization of the document replacement operation to disk even in case that the waitForSync flag had been disabled for the entire collection. Thus, the waitForSync parameter can be used to force synchronization of just specific operations. To use this, set the waitForSync parameter to true. If the waitForSync parameter is not specified or set to false, then the collection's default waitForSync behavior is applied. The waitForSync parameter cannot be used to disable synchronization for collections that have a default waitForSync value of true.

collection.replaceByExample(document, newValue, waitForSync, limit)

The optional limit parameter can be used to restrict the number of replacements to the specified value. If limit is specified but less than the number of documents in the collection, it is undefined which documents are replaced.

Examples

arangosh> db.example.save({ Hello : "world" });

arangosh> db.example.replaceByExample({ Hello: "world" }, {Hello: "mars"}, false, 5);

show execution results

Update By Example

collection.updateByExample(example, newValue)

Partially updates all documents matching an example with a new document body. Specific attributes in the document body of each document matching the example will be updated with the values from newValue. The document meta-attributes such as _id, _key, _from, _to cannot be updated.

Partial update could also be used to append new fields, if there were no old field with same name.

collection.updateByExample(document, newValue, keepNull, waitForSync)

The optional keepNull parameter can be used to modify the behavior when handling null values. Normally, null values are stored in the database. By setting the keepNull parameter to false, this behavior can be changed so that all attributes in data with null values will be removed from the target document.

The optional waitForSync parameter can be used to force synchronization of the document replacement operation to disk even in case that the waitForSync flag had been disabled for the entire collection. Thus, the waitForSync parameter can be used to force synchronization of just specific operations. To use this, set the waitForSync parameter to true. If the waitForSync parameter is not specified or set to false, then the collection's default waitForSync behavior is applied. The waitForSync parameter cannot be used to disable synchronization for collections that have a default waitForSync value of true.

collection.updateByExample(document, newValue, keepNull, waitForSync, limit)

The optional limit parameter can be used to restrict the number of updates to the specified value. If limit is specified but less than the number of documents in the collection, it is undefined which documents are updated.

collection.updateByExample(document, newValue, options)

Using this variant, the options for the operation can be passed using an object with the following sub-attributes:

- keepNull

- waitForSync

- limit

- mergeObjects

Examples

arangosh> db.example.save({ Hello : "world", foo : "bar" });

arangosh> db.example.updateByExample({ Hello: "world" }, { Hello: "foo", World: "bar" }, false);

arangosh> db.example.byExample({ Hello: "foo" }).toArray()

show execution results

First

collection.first(count)

The first method returns the n first documents from the collection, in order of document insertion/update time.

If called with the count argument, the result is a list of up to count documents. If count is bigger than the number of documents in the collection, then the result will contain as many documents as there are in the collection. The result list is ordered, with the "oldest" documents being positioned at the beginning of the result list.

When called without an argument, the result is the first document from the collection. If the collection does not contain any documents, the result returned is null.

Note: this method is not supported in sharded collections with more than one shard.

Examples

arangosh> db.example.first(1);

show execution results

arangosh> db.example.first();

show execution results

Last

collection.last(count)

The last method returns the n last documents from the collection, in order of document insertion/update time.

If called with the count argument, the result is a list of up to count documents. If count is bigger than the number of documents in the collection, then the result will contain as many documents as there are in the collection. The result list is ordered, with the "latest" documents being positioned at the beginning of the result list.

When called without an argument, the result is the last document from the collection. If the collection does not contain any documents, the result returned is null.

Note: this method is not supported in sharded collections with more than one shard.

Examples

arangosh> db.example.last(2);

show execution results

arangosh> db.example.last(1);

show execution results

Collection type

collection.type()

Returns the type of a collection. Possible values are:

- 2: document collection

- 3: edge collection

Get the Version of ArangoDB

db._version()

Returns the server version string. Note that this is not the version of the database.

Examples

arangosh> require("internal").db._version();

2.8.9

Misc

collection.edges(vertex-id)

Returns all edges connected to the vertex specified by vertex-id.

collection.inEdges(vertex-id)

Returns inbound edges connected to the vertex specified by vertex-id.

collection.outEdges(vertex-id)

Returns outbound edges connected to the vertex specified by vertex-id.

iterates over some elements of a collection  collection.iterate(iterator, options)

Iterates over some elements of the collection and apply the function iterator to the elements. The function will be called with the document as first argument and the current number (starting with 0) as second argument.

options must be an object with the following attributes:

- limit (optional, default none): use at most limit documents.

- probability (optional, default all): a number between 0 and 1. Documents are chosen with this probability.

Examples

arangosh> for (i = -90;  i <= 90;  i += 10) {

........>  for (j = -180;  j <= 180;  j += 10) {

........>    db.example.save({ name : "Name/" + i + "/" + j,

........>                      home : [ i, j ],

........>                      work : [ -i, -j ] });

........>  }

........> }

........>

arangosh> db.example.ensureIndex({ type: "geo", fields: [ "home" ] });

arangosh> items = db.example.getIndexes().map(function(x) { return x.id; });

........> db.example.index(items[1]);

show execution results

edge.setProperty(name, value)

Changes or sets the property name an edges to value.

Database Methods

Document

db._document(document)

This method finds a document given its identifier. It returns the document if the document exists. An error is thrown if no document with the given identifier exists, or if the specified _rev value does not match the current revision of the document.

Note: If the method is executed on the arangod server (e.g. from inside a Foxx application), an immutable document object will be returned for performance reasons. It is not possible to change attributes of this immutable object. To update or patch the returned document, it needs to be cloned/copied into a regular JavaScript object first. This is not necessary if the _document method is called from out of arangosh or from any other client.

db._document(document-handle)

As before. Instead of document a document-handle can be passed as first argument.

Examples

Returns the document:

arangosh> db._document("example/12345");

show execution results

Exists

db._exists(document)

This method determines whether a document exists given its identifier. Instead of returning the found document or an error, this method will return either true or false. It can thus be used for easy existence checks.

No error will be thrown if the sought document or collection does not exist. Still this method will throw an error if used improperly, e.g. when called with a non-document handle.

db._exists(document-handle)

As before, but instead of a document a document-handle can be passed.

Replace

db._replace(document, data)

The method returns a document with the attributes _id, _rev and _oldRev. The attribute _id contains the document handle of the updated document, the attribute _rev contains the document revision of the updated document, the attribute _oldRev contains the revision of the old (now replaced) document.

If there is a conflict, i. e. if the revision of the document does not match the revision in the collection, then an error is thrown.

db._replace(document, data, true)

As before, but in case of a conflict, the conflict is ignored and the old document is overwritten.

db._replace(document, data, true, waitForSync)

The optional waitForSync parameter can be used to force synchronization of the document replacement operation to disk even in case that the waitForSync flag had been disabled for the entire collection. Thus, the waitForSync parameter can be used to force synchronization of just specific operations. To use this, set the waitForSync parameter to true. If the waitForSync parameter is not specified or set to false, then the collection's default waitForSync behavior is applied. The waitForSync parameter cannot be used to disable synchronization for collections that have a default waitForSync value of true.

db._replace(document-handle, data)

As before. Instead of document a document-handle can be passed as first argument.

Examples

Create and replace a document:

arangosh> a1 = db.example.insert({ a : 1 });

arangosh> a2 = db._replace(a1, { a : 2 });

arangosh> a3 = db._replace(a1, { a : 3 });

show execution results

Update

db._update(document, data, overwrite, keepNull, waitForSync)

Updates an existing document. The document must be a document in the current collection. This document is then patched with the data given as second argument. The optional overwrite parameter can be used to control the behavior in case of version conflicts (see below). The optional keepNull parameter can be used to modify the behavior when handling null values. Normally, null values are stored in the database. By setting the keepNull parameter to false, this behavior can be changed so that all attributes in data with null values will be removed from the target document.

The optional waitForSync parameter can be used to force synchronization of the document update operation to disk even in case that the waitForSync flag had been disabled for the entire collection. Thus, the waitForSync parameter can be used to force synchronization of just specific operations. To use this, set the waitForSync parameter to true. If the waitForSync parameter is not specified or set to false, then the collection's default waitForSync behavior is applied. The waitForSync parameter cannot be used to disable synchronization for collections that have a default waitForSync value of true*.

The method returns a document with the attributes _id, _rev and _oldRev. The attribute _id contains the document handle of the updated document, the attribute _rev contains the document revision of the updated document, the attribute _oldRev contains the revision of the old (now replaced) document.

If there is a conflict, i. e. if the revision of the document does not match the revision in the collection, then an error is thrown.

db._update(document, data, true)

As before, but in case of a conflict, the conflict is ignored and the old document is overwritten.

db._update(document-handle, data)

As before. Instead of document a document-handle can be passed as first argument.

Examples

Create and update a document:

arangosh> a1 = db.example.insert({ a : 1 });

arangosh> a2 = db._update(a1, { b : 2 });

arangosh> a3 = db._update(a1, { c : 3 });

show execution results

Remove

db._remove(document)

Removes a document. If there is revision mismatch, then an error is thrown.

db._remove(document, true)

Removes a document. If there is revision mismatch, then mismatch is ignored and document is deleted. The function returns true if the document existed and was deleted. It returns false, if the document was already deleted.

db._remove(document, true, waitForSync)  or  db._remove(document, {overwrite: true or false, waitForSync: true or false})

The optional waitForSync parameter can be used to force synchronization of the document deletion operation to disk even in case that the waitForSync flag had been disabled for the entire collection. Thus, the waitForSync parameter can be used to force synchronization of just specific operations. To use this, set the waitForSync parameter to true. If the waitForSync parameter is not specified or set to false, then the collection's default waitForSync behavior is applied. The waitForSync parameter cannot be used to disable synchronization for collections that have a default waitForSync value of true.

db._remove(document-handle, data)

As before. Instead of document a document-handle can be passed as first argument.

Examples

Remove a document:

arangosh> a1 = db.example.insert({ a : 1 });

arangosh> db._remove(a1);

arangosh> db._remove(a1);

arangosh> db._remove(a1, true);

show execution results

Remove a document with a conflict:

arangosh> a1 = db.example.insert({ a : 1 });

arangosh> a2 = db._replace(a1, { a : 2 });

arangosh> db._remove(a1);

arangosh> db._remove(a1, true);

arangosh> db._document(a1);

show execution results

Remove a document using new signature:

arangosh> db.example.insert({ a:  1 } );

arangosh> db.example.remove("example/11265325374", {overwrite: true, waitForSync: false})

show execution results

Handling Indexes

This is an introduction to ArangoDB's interface for indexes in general.

There are special sections for

- index basics

- working with indexes

- hash indexes

- skiplists

- fulltext indexes

- geo-spatial indexes

- cap constraints

Index basics

Indexes allow fast access to documents, provided the indexed attribute(s) are used in a query. While ArangoDB automatically indexes some system attributes, users are free to create extra indexes on non-system attributes of documents.

User-defined indexes can be created on collection level. Most user-defined indexes can be created by specifying the names of the index attributes. Some index types allow indexing just one attribute (e.g. fulltext index) whereas other index types allow indexing multiple attributes at the same time.

The system attributes  _id ,  _key ,  _from  and  _to  are automatically indexed by ArangoDB, without the user being required to create extra indexes for them.

Therefore, indexing  _id ,  _key ,  _rev ,  _from , and  _to  in a user-defined index is often not required and is currently not supported by ArangoDB.

ArangoDB provides the following index types:

Primary Index

For each collection there will always be a primary index which is a hash index for the document keys ( _key  attribute) of all documents in the collection. The primary index allows quick selection of documents in the collection using either the  _key  or  _id  attributes. It will be used from within AQL queries automatically when performing equality lookups on  _key  or  _id .

There are also dedicated functions to find a document given its  _key  or  _id  that will always make use of the primary index:

db.collection.document("<document-key>");

db._document("<document-id>");

As the primary index is an unsorted hash index, it cannot be used for non-equality range queries or for sorting.

The primary index of a collection cannot be dropped or changed, and there is no mechanism to create user-defined primary indexes.

Edges Index

Every edge collection also has an automatically created edges index. The edges index provides quick access to documents by either their  _from  or  _to  attributes. It can therefore be used to quickly find connections between vertex documents and is invoked when the connecting edges of a vertex are queried.

Edges indexes are used from within AQL when performing equality lookups on  _from  or  _to  values in an edge collections. There are also dedicated functions to find edges given their  _from  or  _to  values that will always make use of the edges index:

db.collection.edges("<from-value>");

db.collection.edges("<to-value>");

db.collection.outEdges("<from-value>");

db.collection.outEdges("<to-value>");

db.collection.inEdges("<from-value>");

db.collection.inEdges("<to-value>");

Internally, the edges index is implemented as a hash index. It can be used for equality lookups, but not for range queries or for sorting. As edges indexes are automatically created for edge collections, it is not possible to create user-defined edges indexes.

An edges index cannot be dropped or changed.

Hash Index

A hash index can be used to quickly find documents with specific attribute values. The hash index is unsorted, so it supports equality lookups but no range queries or sorting.

A hash index can be created on one or multiple document attributes. A hash index will only be used by a query if all index attributes are present in the search condition, and if all attributes are compared using the equality ( == ) operator. Hash indexes are used from within AQL and several query functions, e.g.  byExample ,  firstExample  etc.

Hash indexes can optionally be declared unique, then disallowing saving the same value(s) in the indexed attribute(s). Hash indexes can optionally be sparse.

The different types of hash indexes have the following characteristics:

- unique hash index: all documents in the collection must have different values for the attributes covered by the unique index. Trying to insert a document with the same key value as an already existing document will lead to a unique constraint violation.

This type of index is not sparse. Documents that do not contain the index attributes or that have a value of  null  in the index attribute(s) will still be indexed. A key value of  null  may only occur once in the index, so this type of index cannot be used for optional attributes.

- unique, sparse hash index: all documents in the collection must have different values for the attributes covered by the unique index. Documents in which at least one of the index attributes is not set or has a value of  null  are not included in the index. This type of index can be used to ensure that there are no duplicate keys in the collection for documents which have the indexed attributes set. As the index will exclude documents for which the indexed attributes are  null  or not set, it can be used for optional attributes.

- non-unique hash index: all documents in the collection will be indexed. This type of index is not sparse. Documents that do not contain the index attributes or that have a value of  null  in the index attribute(s) will still be indexed. Duplicate key values can occur and do not lead to unique constraint violations.

- non-unique, sparse hash index: only those documents will be indexed that have all the indexed attributes set to a value other than  null . It can be used for optional attributes.

The amortized complexity of lookup, insert, update, and removal operations in unique hash indexes is O(1).

Non-unique hash indexes have an amortized complexity of O(1) for insert, update, and removal operations. That means non-unique hash indexes can be used on attributes with low cardinality.

If a hash index is created on an attribute that is missing in all or many of the documents, the behavior is as follows:

- if the index is sparse, the documents missing the attribute will not be indexed and not use index memory. These documents will not influence the update or removal performance for the index.

- if the index is non-sparse, the documents missing the attribute will be contained in the index with a key value of  null .

Hash indexes support indexing array values if the index attribute name is extended with a [\*]`.

Skiplist Index

A skiplist is a sorted index structure. It can be used to quickly find documents with specific attribute values, for range queries and for returning documents from the index in sorted order. Skiplists will be used from within AQL and several query functions, e.g.  byExample ,  firstExample  etc.

Skiplist indexes will be used for lookups, range queries and sorting only if either all index attributes are provided in a query, or if a leftmost prefix of the index attributes is specified.

For example, if a skiplist index is created on attributes  value1  and  value2 , the following filter conditions can use the index (note: the  <=  and  >=  operators are intentionally omitted here for the sake of brevity):

FILTER doc.value1 == ...

FILTER doc.value1 < ...

FILTER doc.value1 > ...

FILTER doc.value1 > ... && doc.value1 < ...

FILTER doc.value1 == ... && doc.value2 == ...

FILTER doc.value1 == ... && doc.value2 > ...

FILTER doc.value1 == ... && doc.value2 > ... && doc.value2 < ...

In order to use a skiplist index for sorting, the index attributes must be specified in the  SORT  clause of the query in the same order as they appear in the index definition. Skiplist indexes are always created in ascending order, but they can be used to access the indexed elements in both ascending or descending order. However, for a combined index (an index on multiple attributes) this requires that the sort orders in a single query as specified in the  SORT  clause must be either all ascending (optionally ommitted as ascending is the default) or all descending.

For example, if the skiplist index is created on attributes  value1  and  value2  (in this order), then the following sorts clauses can use the index for sorting:

- SORT value1 ASC, value2 ASC  (and its equivalent  SORT value1, value2 )

- SORT value1 DESC, value2 DESC

- SORT value1 ASC  (and its equivalent  SORT value1 )

- SORT value1 DESC

The following sort clauses cannot make use of the index order, and require an extra sort step:

- SORT value1 ASC, value2 DESC

- SORT value1 DESC, value2 ASC

- SORT value2  (and its equivalent  SORT value2 ASC )

- SORT value2 DESC  (because first indexed attribute  value1  is not used in sort clause)

Note: the latter two sort clauses cannot use the index because the sort clause does not refer to a leftmost prefix of the index attributes.

Skiplists can optionally be declared unique, disallowing saving the same value in the indexed attribute. They can be sparse or non-sparse.

The different types of skiplist indexes have the following characteristics:

- unique skiplist index: all documents in the collection must have different values for the attributes covered by the unique index. Trying to insert a document with the same key value as an already existing document will lead to a unique constraint violation.

This type of index is not sparse. Documents that do not contain the index attributes or that have a value of  null  in the index attribute(s) will still be indexed. A key value of  null  may only occur once in the index, so this type of index cannot be used for optional attributes.

- unique, sparse skiplist index: all documents in the collection must have different values for the attributes covered by the unique index. Documents in which at least one of the index attributes is not set or has a value of  null  are not included in the index. This type of index can be used to ensure that there are no duplicate keys in the collection for documents which have the indexed attributes set. As the index will exclude documents for which the indexed attributes are  null  or not set, it can be used for optional attributes.

- non-unique skiplist index: all documents in the collection will be indexed. This type of index is not sparse. Documents that do not contain the index attributes or that have a value of  null  in the index attribute(s) will still be indexed. Duplicate key values can occur and do not lead to unique constraint violations.

- non-unique, sparse skiplist index: only those documents will be indexed that have all the indexed attributes set to a value other than  null . It can be used for optional attributes.

The operational amortized complexity for skiplist indexes is logarithmically correlated with the number of documents in the index.

Skiplist indexes support indexing array values if the index attribute name is extended with a [\*]`.

Geo Index

Users can create additional geo indexes on one or multiple attributes in collections. A geo index is used to find places on the surface of the earth fast.

The geo index stores two-dimensional coordinates. It can be created on either two separate document attributes (latitude and longitude) or a single array attribute that contains both latitude and longitude. Latitude and longitude must be numeric values.

Th geo index provides operations to find documents with coordinates nearest to a given comparison coordinate, and to find documents with coordinates that are within a specifiable radius around a comparison coordinate.

The geo index is used via dedicated functions in AQL or the simple queries functions, but will not be used for other types of queries or conditions.

Fulltext Index

A fulltext index can be used to find words, or prefixes of words inside documents. A fulltext index can be created on a single attribute only, and will index all words contained in documents that have a textual value in that attribute. Only words with a (specifiable) minimum length are indexed. Word tokenization is done using the word boundary analysis provided by libicu, which is taking into account the selected language provided at server start. Words are indexed in their lower-cased form. The index supports complete match queries (full words) and prefix queries, plus basic logical operations such as  and ,  or  and  not  for combining partial results.

The fulltext index is sparse, meaning it will only index documents for which the index attribute is set and contains a string value. Additionally, only words with a configurable minimum length will be included in the index.

The fulltext index is used via dedicated functions in AQL or the simple queries, but will not be enabled for other types of queries or conditions.

Indexing array values

If an index attribute contains an array, ArangoDB will store the entire array as the index value by default. Accessing individual members of the array via the index is not possible this way.

To make an index insert the individual array members into the index instead of the entire array value, a special array index needs to be created for the attribute. Array indexes can be set up like regular hash or skiplist indexes using the  collection.ensureIndex()  function. To make a hash or skiplist index an array index, the index attribute name needs to be extended with [\*] when creating the index and when filtering in an AQL query using the  IN  operator.

The following example creates an array hash index on the  tags  attribute in a collection named  posts :

db.posts.ensureIndex({ type: "hash", fields: [ "tags[*]" ] });

db.posts.insert({ tags: [ "foobar", "baz", "quux" ] });

This array index can then be used for looking up individual  tags  values from AQL queries via the  IN  operator and the attribute name extension [\*]:

FOR doc IN posts

FILTER 'foobar' IN doc.tags[*]

RETURN doc

The following FILTER conditions will not use the array index:

FILTER 'foobar' IN doc.tags

FILTER doc.tags IN 'foobar'

FILTER doc.tags == 'foobar'

FILTER 'foobar' == doc.tags

It is also possible to create an index on subattributes of array values. This makes sense when the index attribute is an array of objects, e.g.

db.posts.ensureIndex({ type: "hash", fields: [ "tags[*].name" ] });

db.posts.insert({ tags: [ { name: "foobar" }, { name: "baz" }, { name: "quux" } ] });

The following query will then use the array index:

FOR doc IN posts

FILTER 'foobar' IN doc.tags[*].name

RETURN doc

If you store a document having the array which does contain elements not having the subattributes this document will also be indexed with the value  null , which in ArangoDB is equal to attribute not existing.

ArangoDB supports creating array indexes with a single [\*] operator per index attribute. For example, creating an index as follows is not supported:

db.posts.ensureIndex({ type: "hash", fields: [ "tags[*].name[*].value" ] });

Array values will automatically be de-duplicated before being inserted into an array index. For example, if the following document is inserted into the collection, the duplicate array value  bar  will be inserted only once:

db.posts.insert({ tags: [ "foobar", "bar", "bar" ] });

If an array index is declared unique, the de-duplication of array values will happen before inserting the values into the index, so the above insert operation will not necessarily fail. It will fail if the index already contains an instance of the  bar  value, but will succeed if the value  bar  is not already present in the index.

If an array index is declared and you store documents that do not have an array at the specified attribute this document will not be inserted in the index. Hence the following objects will not be indexed:

db.posts.ensureIndex({ type: "hash", fields: [ "tags[*]" ] });

db.posts.insert({ something: "else" });

db.posts.insert({ tags: null });

db.posts.insert({ tags: "this is no array" });

db.posts.insert({ tags: { content: [1, 2, 3] } });

An array index is able to index an explicit  null  value and when queried for it, it will only return those documents having explicitly  null  stored in the array, it will not return any documents that do not have the array at all.

db.posts.ensureIndex({ type: "hash", fields: [ "tags[*]" ] });

db.posts.insert({tags: null}) // Will not be indexed

db.posts.insert({tags: []})  // Will not be indexed

db.posts.insert({tags: [null]}); // Will be indexed for null

db.posts.insert({tags: [null, 1, 2]}); // Will be indexed for null, 1 and 2

Declaring an array index as sparse does not have an effect on the array part of the index, this in particular means that explicit  null  values are also indexed in the sparse version. If an index is combined from an array and a normal attribute the sparsity will apply for the attribute e.g.:

db.posts.ensureIndex({ type: "hash", fields: [ "tags[*]", "name" ], sparse: true });

db.posts.insert({tags: null, name: "alice"}) // Will not be indexed

db.posts.insert({tags: [], name: "alice"}) // Will not be indexed

db.posts.insert({tags: [1, 2, 3]}) // Will not be indexed

db.posts.insert({tags: [1, 2, 3], name: null}) // Will not be indexed

db.posts.insert({tags: [1, 2, 3], name: "alice"})

// Will be indexed for [1, "alice"], [2, "alice"], [3, "alice"]

db.posts.insert({tags: [null], name: "bob"})

// Will be indexed for [null, "bob"]

Please note that filtering using array indexes only works from within AQL queries and only if the query filters on the indexed attribute using the  IN  operator. The other comparison operators ( == ,  != ,  > ,  >= ,  < ,  <= ) currently cannot use array indexes.

Which Index to use when

ArangoDB automatically indexes the  _key  attribute in each collection. There is no need to index this attribute separately. Please note that a document's  _id  attribute is derived from the  _key  attribute, and is thus implicitly indexed, too.

ArangoDB will also automatically create an index on  _from  and  _to  in any edge collection, meaning incoming and outgoing connections can be determined efficiently.

Index types

Users can define additional indexes on one or multiple document attributes. Several different index types are provided by ArangoDB. These indexes have different usage scenarios:

- hash index: provides quick access to individual documents if (and only if) all indexed attributes are provided in the search query. The index will only be used for equality comparisons. It does not support range queries and cannot be used for sorting.

The hash index is a good candidate if all or most queries on the indexed attribute(s) are equality comparisons. The unique hash index provides an amortized complexity of O(1) for insert, update, remove and lookup operations. The non-unique hash index provides O(1) inserts, updates and removes, and will allow looking up documents by index value with amortized O(n) complexity, with n being the number of documents with that index value.

A non-unique hash index on an optional document attribute should be declared sparse so that it will not index documents for which the index attribute is not set.

- skiplist index: skiplists keep the indexed values in an order, so they can be used for equality lookups, range queries and for sorting. For high selectivity attributes, skiplist indexes will have a higher overhead than hash indexes. For low selectivity attributes, skiplist indexes will be more efficient than non-unique hash indexes.

Additionally, skiplist indexes allow more use cases (e.g. range queries, sorting) than hash indexes. Furthermore, they can be used for lookups based on a leftmost prefix of the index attributes.

- geo index: the geo index provided by ArangoDB allows searching for documents within a radius around a two-dimensional earth coordinate (point), or to find documents with are closest to a point. Document coordinates can either be specified in two different document attributes or in a single attribute, e.g.

{ "latitude": 50.9406645, "longitude": 6.9599115 }

or

{ "coords": [ 50.9406645, 6.9599115 ] }

Geo indexes will only be invoked via special functions.

- fulltext index: a fulltext index can be used to index all words contained in a specific attribute of all documents in a collection. Only words with a (specifiable) minimum length are indexed. Word tokenization is done using the word boundary analysis provided by libicu, which is taking into account the selected language provided at server start.

The index supports complete match queries (full words) and prefix queries. Fulltext indexes will only be invoked via special functions.

- cap constraint: the cap constraint provided by ArangoDB indexes documents not to speed up search queries, but to limit (cap) the number or size of documents in a collection. This can be used to prevent collections from growing permanently.

Sparse vs. non-sparse indexes

Hash indexes and skiplist indexes can optionally be created sparse. A sparse index does not contain documents for which at least one of the index attribute is not set or contains a value of  null .

As such documents are excluded from sparse indexes, they may contain fewer documents than their non-sparse counterparts. This enables faster indexing and can lead to reduced memory usage in case the indexed attribute does occur only in some, but not all documents of the collection. Sparse indexes will also reduce the number of collisions in non-unique hash indexes in case non-existing or optional attributes are indexed.

In order to create a sparse index, an object with the attribute  sparse  can be added to the index creation commands:

db.collection.ensureIndex({ type: "hash", fields: [ "attributeName" ], sparse: true });

db.collection.ensureIndex({ type: "hash", fields: [ "attributeName1", "attributeName2" ], sparse: true });

db.collection.ensureIndex({ type: "hash", fields: [ "attributeName" ], unique: true, sparse: true });

db.collection.ensureIndex({ type: "hash", fields: [ "attributeName1", "attributeName2" ], unique: true, sparse: true });

db.collection.ensureIndex({ type: "skiplist", fields: [ "attributeName" ], sparse: true });

db.collection.ensureIndex({ type: "skiplist", fields: [ "attributeName1", "attributeName2" ], sparse: true });

db.collection.ensureIndex({ type: "skiplist", fields: [ "attributeName" ], unique: true, sparse: true });

db.collection.ensureIndex({ type: "skiplist", fields: [ "attributeName1", "attributeName2" ], unique: true, sparse: true });

When not explicitly set, the  sparse  attribute defaults to  false  for new indexes. Other indexes than hash and skiplist do not support sparsity.

As sparse indexes may exclude some documents from the collection, they cannot be used for all types of queries. Sparse hash indexes cannot be used to find documents for which at least one of the indexed attributes has a value of  null . For example, the following AQL query cannot use a sparse index, even if one was created on attribute  attr :

FOR doc In collection

FILTER doc.attr == null

RETURN doc

If the lookup value is non-constant, a sparse index may or may not be used, depending on the other types of conditions in the query. If the optimizer can safely determine that the lookup value cannot be  null , a sparse index may be used. When uncertain, the optimizer will not make use of a sparse index in a query in order to produce correct results.

For example, the following queries cannot use a sparse index on  attr  because the optimizer will not know beforehand whether the values which are compared to  doc.attr  will include  null :

FOR doc In collection

FILTER doc.attr == SOME_FUNCTION(...)

RETURN doc

FOR other IN otherCollection

FOR doc In collection

FILTER doc.attr == other.attr

RETURN doc

Sparse skiplist indexes can be used for sorting if the optimizer can safely detect that the index range does not include  null  for any of the index attributes.

Note that if you intend to use joins it may be clever to use non-sparsity and maybe even uniqueness for that attribute, else all items containing the  null  value will match against each other and thus produce large results.

How ArangoDB uses Indexes

In most cases ArangoDB will use a single index per collection in a given query. AQL queries can use more than one index per collection when multiple FILTER conditions are combined with a logical  OR  and these can be covered by indexes. AQL queries will use a single index per collection when FILTER conditions are combined with logical  AND .

Creating multiple indexes on different attributes of the same collection may give the query optimizer more choices when picking an index. Creating multiple indexes on different attributes can also help in speeding up different queries, with FILTER conditions on different attributes.

It is often beneficial to create an index on more than just one attribute. By adding more attributes to an index, an index can become more selective and thus reduce the number of documents that queries need to process.

ArangoDB's primary indexes, edges indexes and hash indexes will automatically provide selectivity estimates. Index selectivity estimates are provided in the web interface, the  getIndexes()  return value and in the  explain()  output for a given query.

The more selective an index is, the more documents it will filter on average. The index selectivity estimates are therefore used by the optimizer when creating query execution plans when there are multiple indexes the optimizer can choose from. The optimizer will then select a combination of indexes with the lowest estimated total cost. In general, the optimizer will pick the indexes with the highest estimated selectivity.

Sparse indexes may or may not be picked by the optimizer in a query. As sparse indexes do not contain  null  values, they will not be used for queries if the optimizer cannot safely determine whether a FILTER condition includes  null  values for the index attributes. The optimizer policy is to produce correct results, regardless of whether or which index is used to satisfy FILTER conditions. If it is unsure about whether using an index will violate the policy, it will not make use of the index.

Troubleshooting

When in doubt about whether and which indexes will be used for executing a given AQL query, use the  explain()  method for the statement as follows (from the ArangoShell):

var query = "FOR doc IN collection FILTER doc.value > 42 RETURN doc";

var stmt = db._createStatement(query);

stmt.explain();

The  explain()  command will return a detailed JSON representation of the query's execution plan. The JSON explain output is intended to be used by code. To get a human-readable and much more compact explanation of the query, there is an explainer tool:

var query = "FOR doc IN collection FILTER doc.value > 42 RETURN doc";

require("org/arangodb/aql/explainer").explain(query);

If any of the explain methods shows that a query is not using indexes, the following steps may help:

- check if the attribute names in the query are correctly spelled. In a schema-free database, documents in the same collection can have varying structures. There is no such thing as a non-existing attribute error. A query that refers to attribute names not present in any of the documents will not return an error, and obviously will not benefit from indexes.

- check the return value of the  getIndexes()  method for the collections used in the query and validate that indexes are actually present on the attributes used in the query's filter conditions.

- if indexes are present but not used by the query, the indexes may have the wrong type. For example, a hash index will only be used for equality comparisons (i.e.  == ) but not for other comparison types such as  < ,  <= ,  > ,  >= . Additionally hash indexes will only be used if all of the index attributes are used in the query's FILTER conditions. A skiplist index will only be used if at least its first attribute is used in a FILTER condition. If additionally of the skiplist index attributes are specified in the query (from left-to-right), they may also be used and allow to filter more documents.

- using indexed attributes as function parameters or in arbitrary expressions will likely lead to the index on the attribute not being used. For example, the following queries will not use an index on  value :

FOR doc IN collection FILTER TO_NUMBER(doc.value) == 42 RETURN doc

FOR doc IN collection FILTER doc.value - 1 == 42 RETURN doc

In these cases the queries should be rewritten so that only the index attribute is present on one side of the operator, or additional filters and indexes should be used to restrict the amount of documents otherwise.

- the query optimizer will in general pick one index per collection in a query. It can pick more than one index per collection if the FILTER condition contains multiple branches combined with logical  OR . For example, the following queries can use indexes:

FOR doc IN collection FILTER doc.value1 == 42 || doc.value1 == 23 RETURN doc

FOR doc IN collection FILTER doc.value1 == 42 || doc.value2 == 23 RETURN doc

FOR doc IN collection FILTER doc.value1 < 42 || doc.value2 > 23 RETURN doc

The two  OR s in the first query will be converted to an  IN  list, and if there is a suitable index on  value1 , it will be used. The second query requires two separate indexes on  value1  and  value2  and will use them if present. The third query can use the indexes on  value1  and  value2  when they are sorted.

Working with Indexes

Index Identifiers and Handles

An index handle uniquely identifies an index in the database. It is a string and consists of the collection name and an index identifier separated by a  / . The index identifier part is a numeric value that is auto-generated by ArangoDB.

A specific index of a collection can be accessed using its index handle or index identifier as follows:

db.collection.index("<index-handle>");

db.collection.index("<index-identifier>");

db._index("<index-handle>");

For example: Assume that the index handle, which is stored in the  _id  attribute of the index, is  demo/362549736  and the index was created in a collection named  demo . Then this index can be accessed as:

db.demo.index("demo/362549736");

Because the index handle is unique within the database, you can leave out the collection and use the shortcut:

db._index("demo/362549736");

Collection Methods

Listing all indexes of a collection

getIndexes()

Returns an array of all indexes defined for the collection.

Note that  _key  implicitly has an index assigned to it.

arangosh> db.test.getIndexes();

show execution results

Creating an index

Indexes should be created using the general method ensureIndex. This method obsoletes the specialized index-specific methods ensureHashIndex, ensureSkiplist, ensureUniqueConstraint etc.

collection.ensureIndex(index-description)

Ensures that an index according to the index-description exists. A new index will be created if none exists with the given description.

The index-description must contain at least a type attribute. Other attributes may be necessary, depending on the index type.

type can be one of the following values:

- hash: hash index

- skiplist: skiplist index

- fulltext: fulltext index

- geo1: geo index, with one attribute

- geo2: geo index, with two attributes

- cap: cap constraint

sparse can be true or false.

For hash, and skiplist the sparsity can be controlled, fulltext and geo are sparse by definition.

unique can be true or false and is supported by hash or skiplist

Calling this method returns an index object. Whether or not the index object existed before the call is indicated in the return attribute isNewlyCreated.

Examples

arangosh> db.test.ensureIndex({ type: "hash", fields: [ "a" ], sparse: true });

arangosh> db.test.ensureIndex({ type: "hash", fields: [ "a", "b" ], unique: true });

show execution results

Dropping an index

collection.dropIndex(index)

Drops the index. If the index does not exist, then false is returned. If the index existed and was dropped, then true is returned. Note that you cannot drop some special indexes (e.g. the primary index of a collection or the edge index of an edge collection).

collection.dropIndex(index-handle)

Same as above. Instead of an index an index handle can be given.

arangosh> db.example.ensureSkiplist("a", "b");

arangosh> var indexInfo = db.example.getIndexes();

arangosh> indexInfo;

arangosh> db.example.dropIndex(indexInfo[0])

arangosh> db.example.dropIndex(indexInfo[1].id)

arangosh> indexInfo = db.example.getIndexes();

show execution results

Database Methods

Fetching an index by handle

db._index(index-handle)

Returns the index with index-handle or null if no such index exists.

arangosh> db.example.ensureIndex({ type: "skiplist", fields: [ "a", "b" ] });

arangosh> var indexInfo = db.example.getIndexes().map(function(x) { return x.id; });

arangosh> indexInfo;

arangosh> db._index(indexInfo[0])

arangosh> db._index(indexInfo[1])

show execution results

Dropping an index

db._dropIndex(index)

Drops the index. If the index does not exist, then false is returned. If the index existed and was dropped, then true is returned.

db._dropIndex(index-handle)

Drops the index with index-handle.

arangosh> db.example.ensureIndex({ type: "skiplist", fields: [ "a", "b" ] });

arangosh> var indexInfo = db.example.getIndexes();

arangosh> indexInfo;

arangosh> db._dropIndex(indexInfo[0])

arangosh> db._dropIndex(indexInfo[1].id)

arangosh> indexInfo = db.example.getIndexes();

show execution results

Revalidating whether an index is used

So you've created an index, and since its maintainance isn't for free, you definitely want to know whether your query can utilize it.

You can use explain to verify whether skiplists or hash indexes are used (if you omit  colors: false  you will get nice colors in ArangoShell):

arangosh> var explain = require("org/arangodb/aql/explainer").explain;

arangosh> db.example.ensureIndex({ type: "skiplist", fields: [ "a", "b" ] });

arangosh> explain("FOR doc IN example FILTER doc.a < 23 RETURN doc", {colors:false});

show execution results

Hash Indexes

Introduction to Hash Indexes

This is an introduction to ArangoDB's hash indexes.

It is possible to define a hash index on one or more attributes (or paths) of a document. This hash index is then used in queries to locate documents in O(1) operations. If the hash index is unique, then no two documents are allowed to have the same set of attribute values.

Creating a new document or updating a document will fail if the uniqueness is violated. If the index is declared sparse, a document will be excluded from the index and no uniqueness checks will be performed if any index attribute value is not set or has a value of  null .

Accessing Hash Indexes from the Shell

collection.ensureIndex({ type: "hash", fields: [ "field1", ..., "fieldn" ], unique: true })

Creates a unique hash index on all documents using field1, ... fieldn as attribute paths. At least one attribute path has to be given. The index will be non-sparse by default.

All documents in the collection must differ in terms of the indexed attributes. Creating a new document or updating an existing document will will fail if the attribute uniqueness is violated.

To create a sparse unique index, set the sparse attribute to  true :

collection.ensureIndex({ type: "hash", fields: [ "field1", ..., "fieldn" ], unique: true, sparse: true })

In case that the index was successfully created, the index identifier is returned.

Non-existing attributes will default to  null . In a sparse index all documents will be excluded from the index for which all specified index attributes are  null . Such documents will not be taken into account for uniqueness checks.

In a non-sparse index, all documents regardless of  null  - attributes will be indexed and will be taken into account for uniqueness checks.

In case that the index was successfully created, an object with the index details, including the index-identifier, is returned.

arangosh> db.test.ensureIndex({ type: "hash", fields: [ "a", "b.c" ], unique: true });

arangosh> db.test.save({ a : 1, b : { c : 1 } });

arangosh> db.test.save({ a : 1, b : { c : 1 } });

arangosh> db.test.save({ a : 1, b : { c : null } });

arangosh> db.test.save({ a : 1 });

show execution results

collection.ensureIndex({ type: "hash", fields: [ "field1", ..., "fieldn" ] })

Creates a non-unique hash index on all documents using field1, ... fieldn as attribute paths. At least one attribute path has to be given. The index will be non-sparse by default.

To create a sparse unique index, set the sparse attribute to  true :

collection.ensureIndex({ type: "hash", fields: [ "field1", ..., "fieldn" ], sparse: true })

In case that the index was successfully created, an object with the index details, including the index-identifier, is returned.

arangosh> db.test.ensureIndex({ type: "hash", fields: [ "a" ] });

arangosh> db.test.save({ a : 1 });

arangosh> db.test.save({ a : 1 });

arangosh> db.test.save({ a : null });

show execution results

Skip-Lists

Introduction to Skiplist Indexes

This is an introduction to ArangoDB's skiplists.

It is possible to define a skiplist index on one or more attributes (or paths) of a documents. This skiplist is then used in queries to locate documents within a given range. If the skiplist is unique, then no two documents are allowed to have the same set of attribute values.

Creating a new document or updating a document will fail if the uniqueness is violated. If the skiplist index is declared sparse, a document will be excluded from the index and no uniqueness checks will be performed if any index attribute value is not set or has a value of  null .

Accessing Skiplist Indexes from the Shell

collection.ensureIndex({ type: "skiplist", fields: [ "field1", ..., "fieldn" ], unique: true })

Creates a unique skiplist index on all documents using field1, ... fieldn as attribute paths. At least one attribute path has to be given. The index will be non-sparse by default.

All documents in the collection must differ in terms of the indexed attributes. Creating a new document or updating an existing document will will fail if the attribute uniqueness is violated.

To create a sparse unique index, set the sparse attribute to  true :

collection.ensureIndex({ type: "skiplist", fields: [ "field1", ..., "fieldn" ], unique: true, sparse: true })

In a sparse index all documents will be excluded from the index that do not contain at least one of the specified index attributes or that have a value of  null  in any of the specified index attributes. Such documents will not be indexed, and not be taken into account for uniqueness checks.

In a non-sparse index, these documents will be indexed (for non-present indexed attributes, a value of  null  will be used) and will be taken into account for uniqueness checks.

In case that the index was successfully created, an object with the index details, including the index-identifier, is returned.

arangosh> db.ids.ensureIndex({ type: "skiplist", fields: [ "myId" ], unique: true });

arangosh> db.ids.save({ "myId": 123 });

arangosh> db.ids.save({ "myId": 456 });

arangosh> db.ids.save({ "myId": 789 });

arangosh> db.ids.save({ "myId": 123 });

show execution results

arangosh> db.ids.ensureIndex({ type: "skiplist", fields: [ "name.first", "name.last" ], unique: true });

arangosh> db.ids.save({ "name" : { "first" : "hans", "last": "hansen" }});

arangosh> db.ids.save({ "name" : { "first" : "jens", "last": "jensen" }});

arangosh> db.ids.save({ "name" : { "first" : "hans", "last": "jensen" }});

show execution results

collection.ensureIndex({ type: "skiplist", fields: [ "field1", ..., "fieldn" ] })

Creates a non-unique skiplist index on all documents using field1, ... fieldn as attribute paths. At least one attribute path has to be given. The index will be non-sparse by default.

To create a sparse unique index, set the sparse attribute to  true .

In case that the index was successfully created, an object with the index details, including the index-identifier, is returned.

arangosh> db.names.ensureIndex({ type: "skiplist", fields: [ "first" ] });

arangosh> db.names.save({ "first" : "Tim" });

arangosh> db.names.save({ "first" : "Tom" });

arangosh> db.names.save({ "first" : "John" });

arangosh> db.names.save({ "first" : "Tim" });

arangosh> db.names.save({ "first" : "Tom" });

show execution results

Query by example using a skiplist index

collection.byExampleSkiplist(index, example)

Selects all documents from the specified skiplist index that match the specified example and returns a cursor.

You can use toArray, next, or hasNext to access the result. The result can be limited using the skip and limit operator.

An attribute name of the form a.b is interpreted as attribute path, not as attribute. If you use

{ a : { c : 1 } }

as example, then you will find all documents, such that the attribute a contains a document of the form {c : 1 }. For example the document

{ a : { c : 1 }, b : 1 }

will match, but the document

{ a : { c : 1, b : 1 } }

will not.

However, if you use

{ a.c : 1 },

then you will find all documents, which contain a sub-document in a that has an attribute @LIT{c} of value 1. Both the following documents

{ a : { c : 1 }, b : 1 }and

{ a : { c : 1, b : 1 } }

will match.

collection.byExampleSkiplist(index, path1, value1, ...)

Fulltext indexes

Introduction to Fulltext Indexes

This is an introduction to ArangoDB's fulltext indexes.

A fulltext index can be used to find words, or prefixes of words inside documents.

A fulltext index can be defined on one attribute only, and will include all words contained in documents that have a textual value in the index attribute. Since ArangoDB 2.6 the index will also include words from the index attribute if the index attribute is an array of strings, or an object with string value members.

For example, given a fulltext index on the  translations  attribute and the following documents, then searching for  лиса  using the fulltext index would return only the first document. Searching for the index for the exact string  Fox  would return the first two documents, and searching for  prefix:Fox  would return all three documents:

{ translations: { en: "fox", de: "Fuchs", fr: "renard", ru: "лиса" } }

{ translations: "Fox is the English translation of the German word Fuchs" }

{ translations: [ "ArangoDB", "document", "database", "Foxx" ] }

If the index attribute is neither a string, an object or an array, its contents will not be indexed. When indexing the contents of an array attribute, an array member will only be included in the index if it is a string. When indexing the contents of an object attribute, an object member value will only be included in the index if it is a string. Other data types are ignored and not indexed.

Accessing Fulltext Indexes from the Shell

collection.ensureIndex({ type: "fulltext", fields: [ "field" ], minLength: minLength })

Creates a fulltext index on all documents on attribute field.

Fulltext indexes are implicitly sparse: all documents which do not have the specified field attribute or that have a non-qualifying value in their field attribute will be ignored for indexing.

Only a single attribute can be indexed. Specifying multiple attributes is unsupported.

The minimum length of words that are indexed can be specified via the minLength parameter. Words shorter than minLength characters will not be indexed. minLength has a default value of 2, but this value might be changed in future versions of ArangoDB. It is thus recommended to explicitly specify this value.

In case that the index was successfully created, an object with the index details is returned.

arangosh> db.example.ensureIndex({ type: "fulltext", fields: [ "text" ], minLength: 3 });

arangosh> db.example.save({ text : "the quick brown", b : { c : 1 } });

arangosh> db.example.save({ text : "quick brown fox", b : { c : 2 } });

arangosh> db.example.save({ text : "brown fox jums", b : { c : 3 } });

arangosh> db.example.save({ text : "fox jumps over", b : { c : 4 } });

arangosh> db.example.save({ text : "jumps over the", b : { c : 5 } });

arangosh> db.example.save({ text : "over the lazy", b : { c : 6 } });

arangosh> db.example.save({ text : "the lazy dog", b : { c : 7 } });

arangosh> db._query("FOR document IN FULLTEXT(example, 'text', 'the') RETURN document");

show execution results

collection.lookupFulltextIndex(attribute, minLength)

Checks whether a fulltext index on the given attribute attribute exists.

Geo Indexes

Introduction to Geo Indexes

This is an introduction to ArangoDB's geo indexes.

ArangoDB uses Hilbert curves to implement geo-spatial indexes. See this blog for details.

A geo-spatial index assumes that the latitude is between -90 and 90 degree and the longitude is between -180 and 180 degree. A geo index will ignore all documents which do not fulfill these requirements.

Accessing Geo Indexes from the Shell

collection.ensureIndex({ type: "geo", fields: [ "location" ] })

Creates a geo-spatial index on all documents using location as path to the coordinates. The value of the attribute has to be an array with at least two numeric values. The array must contain the latitude (first value) and the longitude (second value).

All documents, which do not have the attribute path or have a non-conforming value in it are excluded from the index.

A geo index is implicitly sparse, and there is no way to control its sparsity.

In case that the index was successfully created, an object with the index details, including the index-identifier, is returned.

To create a geo on an array attribute that contains longitude first, set the geoJson attribute to  true . This corresponds to the format described in positions

collection.ensureIndex({ type: "geo", fields: [ "location" ], geoJson: true })

To create a geo-spatial index on all documents using latitude and longitude as separate attribute paths, two paths need to be specified in the fields array:

collection.ensureIndex({ type: "geo", fields: [ "latitude", "longitude" ] })

In case that the index was successfully created, an object with the index details, including the index-identifier, is returned.

Examples

Create a geo index for an array attribute:

arangosh> db.geo.ensureIndex({ type: "geo", fields: [ "loc" ] });

arangosh> for (i = -90;  i <= 90;  i += 10) {

........>     for (j = -180; j <= 180; j += 10) {

........>         db.geo.save({ name : "Name/" + i + "/" + j, loc: [ i, j ] });

........>     }

........> }

arangosh> db.geo.count();

arangosh> db.geo.near(0, 0).limit(3).toArray();

arangosh> db.geo.near(0, 0).count();

show execution results

Create a geo index for a hash array attribute:

arangosh> db.geo2.ensureIndex({ type: "geo", fields: [ "location.latitude", "location.longitude" ] });

arangosh> for (i = -90;  i <= 90;  i += 10) {

........>     for (j = -180; j <= 180; j += 10) {

........>         db.geo2.save({ name : "Name/" + i + "/" + j, location: { latitude : i, longitude : j } });

........>     }

........> }

arangosh> db.geo2.near(0, 0).limit(3).toArray();

show execution results

collection.geo(location-attribute)

Looks up a geo index defined on attribute location_attribute.

Returns a geo index object if an index was found. The  near  or  within  operators can then be used to execute a geo-spatial query on this particular index.

This is useful for collections with multiple defined geo indexes.

collection.geo(location_attribute, true)

Looks up a geo index on a compound attribute location_attribute.

Returns a geo index object if an index was found. The  near  or  within  operators can then be used to execute a geo-spatial query on this particular index.

collection.geo(latitude_attribute, longitude_attribute)

Looks up a geo index defined on the two attributes latitude_attribute and longitude-attribute.

Returns a geo index object if an index was found. The  near  or  within  operators can then be used to execute a geo-spatial query on this particular index.

Note: the geo simple query helper function is deprecated as of ArangoDB 2.6. The function may be removed in future versions of ArangoDB. The preferred way for running geo queries is to use their AQL equivalents.

Examples

Assume you have a location stored as list in the attribute home and a destination stored in the attribute work. Then you can use the  geo  operator to select which geo-spatial attributes (and thus which index) to use in a  near  query.

arangosh> for (i = -90;  i <= 90;  i += 10) {

........>  for (j = -180;  j <= 180;  j += 10) {

........>    db.complex.save({ name : "Name/" + i + "/" + j,

........>                      home : [ i, j ],

........>                      work : [ -i, -j ] });

........>  }

........> }

........>

arangosh> db.complex.near(0, 170).limit(5);

arangosh> db.complex.ensureIndex({ type: "geo", fields: [ "home" ] });

arangosh> db.complex.near(0, 170).limit(5).toArray();

arangosh> db.complex.geo("work").near(0, 170).limit(5);

arangosh> db.complex.ensureIndex({ type: "geo", fields: [ "work" ] });

arangosh> db.complex.geo("work").near(0, 170).limit(5).toArray();

show execution results

collection.near(latitude, longitude)

The returned list is sorted according to the distance, with the nearest document to the coordinate (latitude, longitude) coming first. If there are near documents of equal distance, documents are chosen randomly from this set until the limit is reached. It is possible to change the limit using the limit operator.

In order to use the near operator, a geo index must be defined for the collection. This index also defines which attribute holds the coordinates for the document. If you have more then one geo-spatial index, you can use the geo operator to select a particular index.

Note:  near  does not support negative skips. // However, you can still use  limit  followed to skip.

collection.near(latitude, longitude).limit(limit)

Limits the result to limit documents instead of the default 100.

Note: Unlike with multiple explicit limits,  limit  will raise the implicit default limit imposed by  within .

collection.near(latitude, longitude).distance()

This will add an attribute  distance  to all documents returned, which contains the distance between the given point and the document in meters.

collection.near(latitude, longitude).distance(name)

This will add an attribute name to all documents returned, which contains the distance between the given point and the document in meters.

Note: the near simple query function is deprecated as of ArangoDB 2.6. The function may be removed in future versions of ArangoDB. The preferred way for retrieving documents from a collection using the near operator is to use the AQL NEAR function in an AQL query as follows:

FOR doc IN NEAR(@@collection, @latitude, @longitude, @limit)

RETURN doc

Examples

To get the nearest two locations:

arangosh> db.geo.ensureIndex({ type: "geo", fields: [ "loc" ] });

arangosh> for (var i = -90;  i <= 90;  i += 10) {

........>   for (var j = -180; j <= 180; j += 10) {

........>     db.geo.save({

........>        name : "Name/" + i + "/" + j,

........>        loc: [ i, j ] });

........> } }

arangosh> db.geo.near(0, 0).limit(2).toArray();

show execution results

If you need the distance as well, then you can use the  distance  operator:

arangosh> db.geo.ensureIndex({ type: "geo", fields: [ "loc" ] });

arangosh> for (var i = -90;  i <= 90;  i += 10) {

........>  for (var j = -180; j <= 180; j += 10) {

........>     db.geo.save({

........>         name : "Name/" + i + "/" + j,

........>         loc: [ i, j ] });

........> } }

arangosh> db.geo.near(0, 0).distance().limit(2).toArray();

show execution results

collection.within(latitude, longitude, radius)

This will find all documents within a given radius around the coordinate (latitude, longitude). The returned array is sorted by distance, beginning with the nearest document.

In order to use the within operator, a geo index must be defined for the collection. This index also defines which attribute holds the coordinates for the document. If you have more then one geo-spatial index, you can use the  geo  operator to select a particular index.

collection.within(latitude, longitude, radius).distance()

This will add an attribute  _distance  to all documents returned, which contains the distance between the given point and the document in meters.

collection.within(latitude, longitude, radius).distance(name)

This will add an attribute name to all documents returned, which contains the distance between the given point and the document in meters.

Note: the within simple query function is deprecated as of ArangoDB 2.6. The function may be removed in future versions of ArangoDB. The preferred way for retrieving documents from a collection using the within operator is to use the AQL WITHIN function in an AQL query as follows:

FOR doc IN WITHIN(@@collection, @latitude, @longitude, @radius, @distanceAttributeName)

RETURN doc

Examples

To find all documents within a radius of 2000 km use:

arangosh> for (var i = -90;  i <= 90;  i += 10) {

........>  for (var j = -180; j <= 180; j += 10) {

........> db.geo.save({ name : "Name/" + i + "/" + j, loc: [ i, j ] }); } }

arangosh> db.geo.within(0, 0, 2000 * 1000).distance().toArray();

show execution results

collection.ensureIndex({ type: "geo", fields: [ "location" ] })

Since ArangoDB 2.5, this method is an alias for ensureGeoIndex since geo indexes are always sparse, meaning that documents that do not contain the index attributes or have non-numeric values in the index attributes will not be indexed.

Cap Constraint

Introduction to Cap Constraints

This is an introduction to ArangoDB's size restrictions aka cap constraints for collections.

It is possible to restrict the size of collections. If you add a document and the size exceeds the limit, then the least recently created or updated document(s) will be dropped. The size of a collection is measured in the number of active documents a collection contains, and optionally in the total size of the active documents' data in bytes.

It is possible to only restrict the number of documents in a collection, or to only restrict the total active data size, or both at the same time. If there are restrictions on both document count and total size, then the first violated constraint will trigger the auto-deletion of "too" old documents until all constraints are satisfied.

Using a cap constraint, a collection can be used as a FIFO container, with just the newest documents remaining in the collection.

For example, a cap constraint can be used to keep a list of just the most recent log entries, and at the same time ensure that the collection does not grow indefinitely. Cap constraints can be used to automate the process of getting rid of "old" documents, and so save the user from implementing own jobs to purge "old" collection data.

Accessing Cap Constraints from the Shell

collection.ensureIndex({ type: "cap", size: size, byteSize: byteSize })

Creates a size restriction aka cap for the collection of  size  documents and/or  byteSize  data size. If the restriction is in place and the ( size  plus one) document is added to the collection, or the total active data size in the collection exceeds  byteSize , then the least recently created or updated documents are removed until all constraints are satisfied.

It is allowed to specify either  size  or  byteSize , or both at the same time. If both are specified, then the automatic document removal will be triggered by the first non-met constraint.

Note that at most one cap constraint is allowed per collection. Trying to create additional cap constraints will result in an error. Creating cap constraints is also not supported in sharded collections with more than one shard.

Note that this does not imply any restriction of the number of revisions of documents.

Examples

Restrict the number of document to at most 10 documents:

arangosh> db.examples.ensureIndex({ type: "cap", size: 10 });

arangosh> for (var i = 0;  i < 20;  ++i) { var d = db.examples.save( { n : i } ); }

arangosh> db.examples.count();

show execution results

Simple Queries

Simple queries can be used if the query condition is straight forward, i.e., a document reference, all documents, a query-by-example, or a simple geo query. In a simple query you can specify exactly one collection and one query criteria. In the following sections we describe the JavaScript shell interface for simple queries, which you can use within the ArangoDB shell and within actions and transactions. For other languages see the corresponding language API documentation.

You can find a list of queries at Collection Methods.

Sequential Access and Cursors

If a query returns a cursor, then you can use hasNext and next to iterate over the result set or toArray to convert it to an array.

If the number of query results is expected to be big, it is possible to limit the amount of documents transferred between the server and the client to a specific value. This value is called batchSize. The batchSize can optionally be set before or when a simple query is executed. If the server has more documents than should be returned in a single batch, the server will set the hasMore attribute in the result. It will also return the id of the server-side cursor in the id attribute in the result. This id can be used with the cursor API to fetch any outstanding results from the server and dispose the server-side cursor afterwards.

The initial batchSize value can be set using the setBatchSize method that is available for each type of simple query, or when the simple query is executed using its execute method. If no batchSize value is specified, the server will pick a reasonable default value.

Has Next

cursor.hasNext()

The hasNext operator returns true, then the cursor still has documents. In this case the next document can be accessed using the next operator, which will advance the cursor.

Examples

arangosh> var a = db.five.all();

arangosh> while (a.hasNext()) print(a.next());

show execution results

Next

cursor.next()

If the hasNext operator returns true, then the underlying cursor of the simple query still has documents. In this case the next document can be accessed using the next operator, which will advance the underlying cursor. If you use next on an exhausted cursor, then undefined is returned.

Examples

arangosh> db.five.all().next();

show execution results

Set Batch size

cursor.setBatchSize(number)

Sets the batch size for queries. The batch size determines how many results are at most transferred from the server to the client in one chunk.

Get Batch size

cursor.getBatchSize()

Returns the batch size for queries. If the returned value is undefined, the server will determine a sensible batch size for any following requests.

Execute Query

query.execute(batchSize)

Executes a simple query. If the optional batchSize value is specified, the server will return at most batchSize values in one roundtrip. The batchSize cannot be adjusted after the query is first executed.

Note: There is no need to explicitly call the execute method if another means of fetching the query results is chosen. The following two approaches lead to the same result:

arangosh> result = db.users.all().toArray();

arangosh> q = db.users.all(); q.execute(); result = [ ]; while (q.hasNext()) { result.push(q.next()); }

show execution results

The following two alternatives both use a batchSize and return the same result:

arangosh> q = db.users.all(); q.setBatchSize(20); q.execute(); while (q.hasNext()) { print(q.next()); }

arangosh> q = db.users.all(); q.execute(20); while (q.hasNext()) { print(q.next()); }

show execution results

Dispose

cursor.dispose()

If you are no longer interested in any further results, you should call dispose in order to free any resources associated with the cursor. After calling dispose you can no longer access the cursor.

Count

cursor.count()

The count operator counts the number of document in the result set and returns that number. The count operator ignores any limits and returns the total number of documents found.

Note: Not all simple queries support counting. In this case null is returned.

cursor.count(true)

If the result set was limited by the limit operator or documents were skiped using the skip operator, the count operator with argument true will use the number of elements in the final result set - after applying limit and skip.

Note: Not all simple queries support counting. In this case null is returned.

Examples

Ignore any limit:

arangosh> db.five.all().limit(2).count();

null

Counting any limit or skip:

arangosh> db.five.all().limit(2).count(true);

2

Pagination

If, for example, you display the result of a user search, then you are in general not interested in the completed result set, but only the first 10 or so documents. Or maybe the next 10 documents for the second page. In this case, you can the skip and limit operators. These operators work like LIMIT in MySQL.

skip used together with limit can be used to implement pagination. The skip operator skips over the first n documents. So, in order to create result pages with 10 result documents per page, you can use skip(n * 10).limit(10) to access the 10 documents on the nth page. This result should be sorted, so that the pagination works in a predicable way.

Limit

query.limit(number)

Limits a result to the first number documents. Specifying a limit of 0 will return no documents at all. If you do not need a limit, just do not add the limit operator. The limit must be non-negative.

In general the input to limit should be sorted. Otherwise it will be unclear which documents will be included in the result set.

Examples

arangosh> db.five.all().toArray();

arangosh> db.five.all().limit(2).toArray();

show execution results

Skip

query.skip(number)

Skips the first number documents. If number is positive, then this number of documents are skipped before returning the query results.

In general the input to skip should be sorted. Otherwise it will be unclear which documents will be included in the result set.

Note: using negative skip values is deprecated as of ArangoDB 2.6 and will not be supported in future versions of ArangoDB.

Examples

arangosh> db.five.all().toArray();

arangosh> db.five.all().skip(3).toArray();

show execution results

Modification Queries

ArangoDB also allows removing, replacing, and updating documents based on an example document. Every document in the collection will be compared against the specified example document and be deleted/replaced/ updated if all attributes match.

These method should be used with caution as they are intended to remove or modify lots of documents in a collection.

All methods can optionally be restricted to a specific number of operations. However, if a limit is specific but is less than the number of matches, it will be undefined which of the matching documents will get removed/modified. Remove by Example, Replace by Example and Update by Example are described with examples in the subchapter Collection Methods.

Geo Queries

The ArangoDB allows to select documents based on geographic coordinates. In order for this to work, a geo-spatial index must be defined. This index will use a very elaborate algorithm to lookup neighbors that is a magnitude faster than a simple R* index.

In general a geo coordinate is a pair of latitude and longitude, which must both be specified as numbers. A geo index can be created on coordinates that are stored in a single list attribute with two elements like [-10, +30] (latitude first, followed by longitude) or on coordinates stored in two separate attributes.

For example, to index the following documents, an index can be created on the position attribute of the documents:

db.test.save({ position: [ -10, 30 ] });

db.test.save({ position: [ 10, 45.5 ] });

db.test.ensureIndex({ type: "geo", fields: [ "position" ] });

If coordinates are stored in two distinct attributes, the index must be created on the two attributes:

db.test.save({ latitude: -10, longitude: 30 });

db.test.save({ latitude: 10, longitude: 45.5 });

db.test.ensureIndex({ type: "geo", fields: [ "latitude", "longitude" ] });

In order to find all documents within a given radius around a coordinate use the within operator. In order to find all documents near a given document use the near operator.

It is possible to define more than one geo-spatial index per collection. In this case you must give a hint using the geo operator which of indexes should be used in a query.

Near

collection.near(latitude, longitude)

The returned list is sorted according to the distance, with the nearest document to the coordinate (latitude, longitude) coming first. If there are near documents of equal distance, documents are chosen randomly from this set until the limit is reached. It is possible to change the limit using the limit operator.

In order to use the near operator, a geo index must be defined for the collection. This index also defines which attribute holds the coordinates for the document. If you have more then one geo-spatial index, you can use the geo operator to select a particular index.

Note:  near  does not support negative skips. // However, you can still use  limit  followed to skip.

collection.near(latitude, longitude).limit(limit)

Limits the result to limit documents instead of the default 100.

Note: Unlike with multiple explicit limits,  limit  will raise the implicit default limit imposed by  within .

collection.near(latitude, longitude).distance()

This will add an attribute  distance  to all documents returned, which contains the distance between the given point and the document in meters.

collection.near(latitude, longitude).distance(name)

This will add an attribute name to all documents returned, which contains the distance between the given point and the document in meters.

Note: the near simple query function is deprecated as of ArangoDB 2.6. The function may be removed in future versions of ArangoDB. The preferred way for retrieving documents from a collection using the near operator is to use the AQL NEAR function in an AQL query as follows:

FOR doc IN NEAR(@@collection, @latitude, @longitude, @limit)

RETURN doc

Examples

To get the nearest two locations:

arangosh> db.geo.ensureIndex({ type: "geo", fields: [ "loc" ] });

arangosh> for (var i = -90;  i <= 90;  i += 10) {

........>   for (var j = -180; j <= 180; j += 10) {

........>     db.geo.save({

........>        name : "Name/" + i + "/" + j,

........>        loc: [ i, j ] });

........> } }

arangosh> db.geo.near(0, 0).limit(2).toArray();

show execution results

If you need the distance as well, then you can use the  distance  operator:

arangosh> db.geo.ensureIndex({ type: "geo", fields: [ "loc" ] });

arangosh> for (var i = -90;  i <= 90;  i += 10) {

........>  for (var j = -180; j <= 180; j += 10) {

........>     db.geo.save({

........>         name : "Name/" + i + "/" + j,

........>         loc: [ i, j ] });

........> } }

arangosh> db.geo.near(0, 0).distance().limit(2).toArray();

show execution results

Within

collection.within(latitude, longitude, radius)

This will find all documents within a given radius around the coordinate (latitude, longitude). The returned array is sorted by distance, beginning with the nearest document.

In order to use the within operator, a geo index must be defined for the collection. This index also defines which attribute holds the coordinates for the document. If you have more then one geo-spatial index, you can use the  geo  operator to select a particular index.

collection.within(latitude, longitude, radius).distance()

This will add an attribute  _distance  to all documents returned, which contains the distance between the given point and the document in meters.

collection.within(latitude, longitude, radius).distance(name)

This will add an attribute name to all documents returned, which contains the distance between the given point and the document in meters.

Note: the within simple query function is deprecated as of ArangoDB 2.6. The function may be removed in future versions of ArangoDB. The preferred way for retrieving documents from a collection using the within operator is to use the AQL WITHIN function in an AQL query as follows:

FOR doc IN WITHIN(@@collection, @latitude, @longitude, @radius, @distanceAttributeName)

RETURN doc

Examples

To find all documents within a radius of 2000 km use:

arangosh> for (var i = -90;  i <= 90;  i += 10) {

........>  for (var j = -180; j <= 180; j += 10) {

........> db.geo.save({ name : "Name/" + i + "/" + j, loc: [ i, j ] }); } }

arangosh> db.geo.within(0, 0, 2000 * 1000).distance().toArray();

show execution results

Geo

collection.geo(location-attribute)

Looks up a geo index defined on attribute location_attribute.

Returns a geo index object if an index was found. The  near  or  within  operators can then be used to execute a geo-spatial query on this particular index.

This is useful for collections with multiple defined geo indexes.

collection.geo(location_attribute, true)

Looks up a geo index on a compound attribute location_attribute.

Returns a geo index object if an index was found. The  near  or  within  operators can then be used to execute a geo-spatial query on this particular index.

collection.geo(latitude_attribute, longitude_attribute)

Looks up a geo index defined on the two attributes latitude_attribute and longitude-attribute.

Returns a geo index object if an index was found. The  near  or  within  operators can then be used to execute a geo-spatial query on this particular index.

Note: the geo simple query helper function is deprecated as of ArangoDB 2.6. The function may be removed in future versions of ArangoDB. The preferred way for running geo queries is to use their AQL equivalents.

Examples

Assume you have a location stored as list in the attribute home and a destination stored in the attribute work. Then you can use the  geo  operator to select which geo-spatial attributes (and thus which index) to use in a  near  query.

arangosh> for (i = -90;  i <= 90;  i += 10) {

........>  for (j = -180;  j <= 180;  j += 10) {

........>    db.complex.save({ name : "Name/" + i + "/" + j,

........>                      home : [ i, j ],

........>                      work : [ -i, -j ] });

........>  }

........> }

........>

arangosh> db.complex.near(0, 170).limit(5);

arangosh> db.complex.ensureIndex({ type: "geo", fields: [ "home" ] });

arangosh> db.complex.near(0, 170).limit(5).toArray();

arangosh> db.complex.geo("work").near(0, 170).limit(5);

arangosh> db.complex.ensureIndex({ type: "geo", fields: [ "work" ] });

arangosh> db.complex.geo("work").near(0, 170).limit(5).toArray();

show execution results

Fulltext queries

ArangoDB allows to run queries on text contained in document attributes. To use this, a fulltext index must be defined for the attribute of the collection that contains the text. Creating the index will parse the text in the specified attribute for all documents of the collection. Only documents will be indexed that contain a textual value in the indexed attribute. For such documents, the text value will be parsed, and the individual words will be inserted into the fulltext index.

When a fulltext index exists, it can be queried using a fulltext query.

Fulltext

collection.fulltext(attribute, query)

The fulltext simple query functions performs a fulltext search on the specified attribute and the specified query.

Details about the fulltext query syntax can be found below.

Note: the fulltext simple query function is deprecated as of ArangoDB 2.6. The function may be removed in future versions of ArangoDB. The preferred way for executing fulltext queries is to use an AQL query using the FULLTEXT AQL function as follows:

FOR doc IN FULLTEXT(@@collection, @attributeName, @queryString, @limit)

RETURN doc

Examples

arangosh> db.emails.ensureFulltextIndex("content");

arangosh> db.emails.save({ content: "Hello Alice, how are you doing? Regards, Bob" });

arangosh> db.emails.save({ content: "Hello Charlie, do Alice and Bob know about it?" });

arangosh> db.emails.save({ content: "I think they don't know. Regards, Eve" });

arangosh> db.emails.fulltext("content", "charlie,|eve").toArray();

show execution results

Fulltext Syntax:

In the simplest form, a fulltext query contains just the sought word. If multiple search words are given in a query, they should be separated by commas. All search words will be combined with a logical AND by default, and only such documents will be returned that contain all search words. This default behavior can be changed by providing the extra control characters in the fulltext query, which are:

- +: logical AND (intersection)

- |: logical OR (union)

- -: negation (exclusion)

Examples:

- "banana": searches for documents containing "banana"

- "banana,apple": searches for documents containing both "banana" AND "apple"

- "banana,|orange": searches for documents containing either "banana" OR "orange" OR both

- "banana,-apple": searches for documents that contains "banana" but NOT "apple".

Logical operators are evaluated from left to right.

Each search word can optionally be prefixed with complete: or prefix:, with complete: being the default. This allows searching for complete words or for word prefixes. Suffix searches or any other forms are partial-word matching are currently not supported.

Examples:

- "complete:banana": searches for documents containing the exact word "banana"

- "prefix:head": searches for documents with words that start with prefix "head"

- "prefix:head,banana": searches for documents contain words starting with prefix "head" and that also contain the exact word "banana".

Complete match and prefix search options can be combined with the logical operators.

Please note that only words with a minimum length will get indexed. This minimum length can be defined when creating the fulltext index. For words tokenization, the libicu text boundary analysis is used, which takes into account the default as defined at server startup (--server.default-language startup option). Generally, the word boundary analysis will filter out punctuation but will not do much more.

Especially no word normalization, stemming, or similarity analysis will be performed when indexing or searching. If any of these features is required, it is suggested that the user does the text normalization on the client side, and provides for each document an extra attribute containing just a comma-separated list of normalized words. This attribute can then be indexed with a fulltext index, and the user can send fulltext queries for this index, with the fulltext queries also containing the stemmed or normalized versions of words as required by the user.

Transactions

Starting with version 1.3, ArangoDB provides support for user-definable transactions.

Transactions in ArangoDB are atomic, consistent, isolated, and durable (ACID).

These ACID properties provide the following guarantees:

- The atomicity principle makes transactions either complete in their entirety or have no effect at all.

- The consistency principle ensures that no constraints or other invariants will be violated during or after any transaction.

- The isolation property will hide the modifications of a transaction from other transactions until the transaction commits.

- Finally, the durability proposition makes sure that operations from transactions that have committed will be made persistent. The amount of transaction durability is configurable in ArangoDB, as is the durability on collection level.

Transaction invocation

ArangoDB transactions are different from transactions in SQL.

In SQL, transactions are started with explicit BEGIN or START TRANSACTION command. Following any series of data retrieval or modification operations, an SQL transaction is finished with a COMMIT command, or rolled back with a ROLLBACK command. There may be client/server communication between the start and the commit/rollback of an SQL transaction.

In ArangoDB, a transaction is always a server-side operation, and is executed on the server in one go, without any client interaction. All operations to be executed inside a transaction need to be known by the server when the transaction is started.

There are no individual BEGIN, COMMIT or ROLLBACK transaction commands in ArangoDB. Instead, a transaction in ArangoDB is started by providing a description of the transaction to the db._executeTransaction JavaScript function:

db._executeTransaction(description);

This function will then automatically start a transaction, execute all required data retrieval and/or modification operations, and at the end automatically commit the transaction. If an error occurs during transaction execution, the transaction is automatically aborted, and all changes are rolled back.

Execute transaction

db._executeTransaction(object)

Executes a server-side transaction, as specified by object.

object must have the following attributes:

- collections: a sub-object that defines which collections will be used in the transaction. collections can have these attributes:

- read: a single collection or a list of collections that will be used in the transaction in read-only mode

- write: a single collection or a list of collections that will be used in the transaction in write or read mode.

- action: a Javascript function or a string with Javascript code containing all the instructions to be executed inside the transaction. If the code runs through successfully, the transaction will be committed at the end. If the code throws an exception, the transaction will be rolled back and all database operations will be rolled back.

Additionally, object can have the following optional attributes:

- waitForSync: boolean flag indicating whether the transaction is forced to be synchronous.

- lockTimeout: a numeric value that can be used to set a timeout for waiting on collection locks. If not specified, a default value will be used. Setting lockTimeout to 0 will make ArangoDB not time out waiting for a lock.

- params: optional arguments passed to the function specified in action.

Declaration of collections

All collections which are to participate in a transaction need to be declared beforehand. This is a necessity to ensure proper locking and isolation.

Collections can be used in a transaction in write mode or in read-only mode.

If any data modification operations are to be executed, the collection must be declared for use in write mode. The write mode allows modifying and reading data from the collection during the transaction (i.e. the write mode includes the read mode).

Contrary, using a collection in read-only mode will only allow performing read operations on a collection. Any attempt to write into a collection used in read-only mode will make the transaction fail.

Collections for a transaction are declared by providing them in the collections attribute of the object passed to the _executeTransaction function. The collections attribute has the sub-attributes read and write:

db._executeTransaction({

collections: {

write: [ "users", "logins" ],

read: [ "recommendations" ]

}

});

read and write are optional attributes, and only need to be specified if the operations inside the transactions demand for it.

The contents of read or write can each be lists arrays collection names or a single collection name (as a string):

db._executeTransaction({

collections: {

write: "users",

read: "recommendations"

}

});

Note: It is currently optional to specify collections for read-only access. Even without specifying them, it is still possible to read from such collections from within a transaction, but with relaxed isolation. Please refer to Transactions Locking for more details.

In order to make a transaction fail when a non-declared collection is used inside for reading, the optional allowImplicit sub-attribute of collections can be set to false:

db._executeTransaction({

collections: {

read: "recommendations",

allowImplicit: false  /* this disallows read access to other collections

than specified */

},

action: function () {

var db = require("org/arangodb").db;

return db.foobar.toArray(); /* will fail because db.foobar must not be accessed

for reading inside this transaction */

}

});

The default value for allowImplicit is true. Write-accessing collections that have not been declared in the collections array is never possible, regardless of the value of allowImplicit.

Declaration of data modification and retrieval operations

All data modification and retrieval operations that are to be executed inside the transaction need to be specified in a Javascript function, using the action attribute:

db._executeTransaction({

collections: {

write: "users"

},

action: function () {

// all operations go here

}

});

Any valid Javascript code is allowed inside action but the code may only access the collections declared in collections. action may be a Javascript function as shown above, or a string representation of a Javascript function:

db._executeTransaction({

collections: {

write: "users"

},

action: "function () { doSomething(); }"

});

Please note that any operations specified in action will be executed on the server, in a separate scope. Variables will be bound late. Accessing any JavaScript variables defined on the client-side or in some other server context from inside a transaction may not work. Instead, any variables used inside action should be defined inside action itself:

db._executeTransaction({

collections: {

write: "users"

},

action: function () {

var db = require(...).db;

db.users.save({ ... });

}

});

When the code inside the action attribute is executed, the transaction is already started and all required locks have been acquired. When the code inside the action attribute finishes, the transaction will automatically commit. There is no explicit commit command.

To make a transaction abort and roll back all changes, an exception needs to be thrown and not caught inside the transaction:

db._executeTransaction({

collections: {

write: "users"

},

action: function () {

var db = require("internal").db;

db.users.save({ _key: "hello" });

// will abort and roll back the transaction

throw "doh!";

}

});

There is no explicit abort or roll back command.

As mentioned earlier, a transaction will commit automatically when the end of the action function is reached and no exception has been thrown. In this case, the user can return any legal JavaScript value from the function:

db._executeTransaction({

collections: {

write: "users"

},

action: function () {

var db = require("internal").db;

db.users.save({ _key: "hello" });

// will commit the transaction and return the value "hello"

return "hello";

}

});

Examples

The first example will write 3 documents into a collection named c1. The c1 collection needs to be declared in the write attribute of the collections attribute passed to the executeTransaction function.

The action attribute contains the actual transaction code to be executed. This code contains all data modification operations (3 in this example).

// setup

db._create("c1");

db._executeTransaction({

collections: {

write: [ "c1" ]

},

action: function () {

var db = require("internal").db;

db.c1.save({ _key: "key1" });

db.c1.save({ _key: "key2" });

db.c1.save({ _key: "key3" });

}

});

db.c1.count(); // 3

Aborting the transaction by throwing an exception in the action function will revert all changes, so as if the transaction never happened:

// setup

db._create("c1");

db._executeTransaction({

collections: {

write: [ "c1" ]

},

action: function () {

var db = require("internal").db;

db.c1.save({ _key: "key1" });

db.c1.count(); // 1

db.c1.save({ _key: "key2" });

db.c1.count(); // 2

throw "doh!";

}

});

db.c1.count(); // 0

The automatic rollback is also executed when an internal exception is thrown at some point during transaction execution:

// setup

db._create("c1");

db._executeTransaction({

collections: {

write: [ "c1" ]

},

action: function () {

var db = require("internal").db;

db.c1.save({ _key: "key1" });

// will throw duplicate a key error, not explicitly requested by the user

db.c1.save({ _key: "key1" });

// we'll never get here...

}

});

db.c1.count(); // 0

As required by the consistency principle, aborting or rolling back a transaction will also restore secondary indexes to the state at transaction start. The following example using a cap constraint should illustrate that:

// setup

db._create("c1");

// limit the number of documents to 3

db.c1.ensureIndex({ type: "cap", size: 3 });

// insert 3 documents

db.c1.save({ _key: "key1" });

db.c1.save({ _key: "key2" });

db.c1.save({ _key: "key3" });

// this will push out key1

db.c1.save({ _key: "key4" });

db._executeTransaction({

collections: {

write: [ "c1" ]

},

action: function () {

var db = require("internal").db;

// this will push out key2

db.c1.save({ _key: "key5" });

// will abort the transaction

throw "doh!"

}

});

// we now have these keys back: [ "key2", "key3", "key4" ]

Cross-collection transactions

There's also the possibility to run a transaction across multiple collections. In this case, multiple collections need to be declared in the collections attribute, e.g.:

// setup

db._create("c1");

db._create("c2");

db._executeTransaction({

collections: {

write: [ "c1", "c2" ]

},

action: function () {

var db = require("internal").db;

db.c1.save({ _key: "key1" });

db.c2.save({ _key: "key2" });

}

});

db.c1.count(); // 1

db.c2.count(); // 1

Again, throwing an exception from inside the action function will make the transaction abort and roll back all changes in all collections:

// setup

db._create("c1");

db._create("c2");

db._executeTransaction({

collections: {

write: [ "c1", "c2" ]

},

action: function () {

var db = require("internal").db;

for (var i = 0; i < 100; ++i) {

db.c1.save({ _key: "key" + i });

db.c2.save({ _key: "key" + i });

}

db.c1.count(); // 100

db.c2.count(); // 100

// abort

throw "doh!"

}

});

db.c1.count(); // 0

db.c2.count(); // 0

Passing parameters to transactions

Arbitrary parameters can be passed to transactions by setting the params attribute when declaring the transaction. This feature is handy to re-use the same transaction code for multiple calls but with different parameters.

A basic example:

db._executeTransaction({

collections: { },

action: "function (params) { return params[1]; }",

params: [ 1, 2, 3 ]

});

The above example will return 2.

Some example that uses collections:

db._executeTransaction({

collections: {

write: "users",

read: [ "c1", "c2" ]

},

action: "function (params) { var db = require('internal').db; var doc = db.c1.document(params['c1Key']); db.users.save(doc); doc = db.c2.document(params['c2Key']); db.users.save(doc);}",

params: {

c1Key: "foo",

c2Key: "bar"

}

});

Disallowed operations

Some operations are not allowed inside ArangoDB transactions:

- creation and deletion of collections (db._create(), db._drop(), db._rename())

- creation and deletion of indexes (db.ensure...Index(), db.dropIndex())

If an attempt is made to carry out any of these operations during a transaction, ArangoDB will abort the transaction with error code 1653 (disallowed operation inside transaction).

Locking and Isolation

All collections specified in the collections attribute are locked in the requested mode (read or write) at transaction start. Locking of multiple collections is performed in alphabetical order. When a transaction commits or rolls back, all locks are released in reverse order. The locking order is deterministic to avoid deadlocks.

While locks are held, modifications by other transactions to the collections participating in the transaction are prevented. A transaction will thus see a consistent view of the participating collections' data.

Additionally, a transaction will not be interrupted or interleaved with any other ongoing operations on the same collection. This means each transaction will run in isolation. A transaction should never see uncommitted or rolled back modifications by other transactions. Additionally, reads inside a transaction are repeatable.

Note that the above is true only for all collections that are declared in the collections attribute of the transaction.

Lazily adding collections

There might be situations when declaring all collections a priori is not possible, for example, because further collections are determined by a dynamic AQL query inside the transaction, for example a query using AQL graph functions.

In this case, it would be impossible to know beforehand which collection to lock, and thus it is legal to not declare collections that will be accessed in the transaction in read-only mode. Accessing a non-declared collection in read-only mode during a transaction will add the collection to the transaction lazily, and fetch data from the collection as usual. However, as the collection ie added lazily, there is no isolation from other concurrent operations or transactions. Reads from such collections are potentially non-repeatable.

EXAMPLES:

db._executeTransaction({

collections: {

read: "users"

},

action: function () {

/* execute an AQL query that traverses a graph starting at a "users" vertex.

it is yet unknown into which other collections the query will traverse */

db._createStatement({

query: "FOR t IN TRAVERSAL(users, connections, "users/1234", "any", { }) RETURN t"

}).execute().toArray().forEach(function (d) {

/* ... */

});

}

});

This automatic lazy addition of collections to a transaction also introduces the possibility of deadlocks. Deadlocks may occur if there are concurrent transactions that try to acquire locks on the same collections lazily.

In order to make a transaction fail when a non-declared collection is used inside a transaction for reading, the optional allowImplicit sub-attribute of collections can be set to false:

db._executeTransaction({

collections: {

read: "users",

allowImplicit: false

},

action: function () {

/* the below query will now fail because the collection connections has not

been specified in the list of collections used by the transaction */

db._createStatement({

query: "FOR t IN TRAVERSAL(users, connections, "users/1234", "any", { }) RETURN t"

}).execute().toArray().forEach(function (d) {

/* ... */

});

}

});

The default value for allowImplicit is true. Write-accessing collections that have not been declared in the collections array is never possible, regardless of the value of allowImplicit.

Deadlocks and Deadlock detection

A deadlock is a situation in which two or more concurrent operations (user transactions or AQL queries) try to access the same resources (collections, documents) and need to wait for the others to finish, but none of them can make any progress.

A good example for a deadlock is two concurrently executing transactions T1 and T2 that try to access the same collections but that need to wait for each other. In this example, transaction T1 will write to collection  c1 , but will also read documents from collection  c2  without announcing it:

db._executeTransaction({

collections: {

write: "c1"

},

action: function () {

var db = require("org/arangodb").db;

/* write into c1 (announced) */

db.c1.insert({ foo: "bar" });

/* some operation here that takes long to execute... */

/* read from c2 (unannounced) */

db.c2.toArray();

}

});

Transaction T2 announces to write into collection  c2 , but will also read documents from collection  c1  without announcing it:

db._executeTransaction({

collections: {

write: "c2"

},

action: function () {

var db = require("org/arangodb").db;

/* write into c2 (announced) */

db.c2.insert({ bar: "baz" });

/* some operation here that takes long to execute... */

/* read from c1 (unannounced) */

db.c1.toArray();

}

});

In the above example, a deadlock will occur if transaction T1 and T2 have both acquired their write locks (T1 for collection  c1  and T2 for collection  c2 ) and are then trying to read from the other other (T1 will read from  c2 , T2 will read from  c1 ). T1 will then try to acquire the read lock on collection  c2 , which is prevented by transaction T2. T2 however will wait for the read lock on collection  c1 , which is prevented by transaction T1.

In case of such deadlock, there would be no progress for any of the involved transactions, and none of the involved transactions could ever complete. This is completely undesirable, so the automatic deadlock detection mechanism in ArangoDB will automatically abort one of the transactions involved in such deadlock. Aborting means that all changes done by the transaction will be rolled back and error 29 ( deadlock detected ) will be thrown.

Client code (AQL queries, user transactions) that accesses more than one collection should be aware of the potential of deadlocks and should handle the error 29 ( deadlock detected ) properly, either by passing the exception to the caller or retrying the operation.

To avoid both deadlocks and non-repeatable reads, all collections used in a transaction should be specified in the  collections  attribute when known in advance. In case this is not possible because collections are added dynamically inside the transaction, deadlocks may occur and the deadlock detection may kick in and abort the transaction.

Durability

Transactions are executed in main memory first until there is either a rollback or a commit. On rollback, no data will be written to disk, but the operations from the transaction will be reversed in memory.

On commit, all modifications done in the transaction will be written to the collection datafiles. These writes will be synchronized to disk if any of the modified collections has the waitForSync property set to true, or if any individual operation in the transaction was executed with the waitForSync attribute. Additionally, transactions that modify data in more than one collection are automatically synchronized to disk. This synchronization is done to not only ensure durability, but to also ensure consistency in case of a server crash.

That means if you only modify data in a single collection, and that collection has its waitForSync property set to false, the whole transaction will not be synchronized to disk instantly, but with a small delay.

There is thus the potential risk of losing data between the commit of the transaction and the actual (delayed) disk synchronization. This is the same as writing into collections that have the waitForSync property set to false outside of a transaction. In case of a crash with waitForSync set to false, the operations performed in the transaction will either be visible completely or not at all, depending on whether the delayed synchronization had kicked in or not.

To ensure durability of transactions on a collection that have the waitForSync property set to false, you can set the waitForSync attribute of the object that is passed to executeTransaction. This will force a synchronization of the transaction to disk even for collections that have waitForSync set to false:

db._executeTransaction({

collections: {

write: "users"

},

waitForSync: true,

action: function () { ... }

});

An alternative is to perform an operation with an explicit sync request in a transaction, e.g.

db.users.save({ _key: "1234" }, true);

In this case, the true value will make the whole transaction be synchronized to disk at the commit.

In any case, ArangoDB will give users the choice of whether or not they want full durability for single collection transactions. Using the delayed synchronization (i.e. waitForSync with a value of false) will potentially increase throughput and performance of transactions, but will introduce the risk of losing the last committed transactions in the case of a crash.

In contrast, transactions that modify data in more than one collection are automatically synchronized to disk. This comes at the cost of several disk sync. For a multi-collection transaction, the call to the _executeTransaction function will only return after the data of all modified collections has been synchronized to disk and the transaction has been made fully durable. This not only reduces the risk of losing data in case of a crash but also ensures consistency after a restart.

In case of a server crash, any multi-collection transactions that were not yet committed or in preparation to be committed will be rolled back on server restart.

For multi-collection transactions, there will be at least one disk sync operation per modified collection. Multi-collection transactions thus have a potentially higher cost than single collection transactions. There is no configuration to turn off disk synchronization for multi-collection transactions in ArangoDB. The disk sync speed of the system will thus be the most important factor for the performance of multi-collection transactions.

Limitations

Transactions in ArangoDB have been designed with particular use cases in mind. They will be mainly useful for short and small data retrieval and/or modification operations.

The implementation is not optimized for very long-running or very voluminous operations, and may not be usable for these cases.

One limitation is that a transaction operation information must fit into main memory. The transaction information consists of record pointers, revision numbers and rollback information. The actual data modification operations of a transaction are written to the write-ahead log and do not need to fit entirely into main memory.

Ongoing transactions will also prevent the write-ahead logs from being fully garbage-collected. Information in the write-ahead log files cannot be written to collection data files or be discarded while transactions are ongoing.

To ensure progress of the write-ahead log garbage collection, transactions should be kept as small as possible, and big transactions should be split into multiple smaller transactions.

Transactions in ArangoDB cannot be nested, i.e. a transaction must not start another transaction. If an attempt is made to call a transaction from inside a running transaction, the server will throw error 1651 (nested transactions detected).

It is also disallowed to execute user transaction on some of ArangoDB's own system collections. This shouldn't be a problem for regular usage as system collections will not contain user data and there is no need to access them from within a user transaction.

Finally, all collections that may be modified during a transaction must be declared beforehand, i.e. using the collections attribute of the object passed to the _executeTransaction function. If any attempt is made to carry out a data modification operation on a collection that was not declared in the collections attribute, the transaction will be aborted and ArangoDB will throw error 1652 unregistered collection used in transaction. It is legal to not declare read-only collections, but this should be avoided if possible to reduce the probability of deadlocks and non-repeatable reads.

Please refer to Locking and Isolation for more details.

Introduction

The ArangoDB query language (AQL) can be used to retrieve and modify data that are stored in ArangoDB. The general workflow when executing a query is as follows:

- A client application ships an AQL query to the ArangoDB server. The query text contains everything ArangoDB needs to compile the result set

- ArangoDB will parse the query, execute it and compile the results. If the query is invalid or cannot be executed, the server will return an error that the client can process and react to. If the query can be executed successfully, the server will return the query results (if any) to the client

AQL is mainly a declarative language, meaning that a query expresses what result should be achieved but not how it should be achieved. AQL aims to be human-readable and therefore uses keywords from the English language. Another design goal of AQL was client independency, meaning that the language and syntax are the same for all clients, no matter what programming language the clients may use. Further design goals of AQL were the support of complex query patterns and the different data models ArangoDB offers.

In its purpose, AQL is similar to the Structured Query Language (SQL). AQL supports reading and modifying collection data, but it doesn't support data-definition operations such as creating and dropping databases, collections and indexes.

The syntax of AQL queries is different to SQL, even if some keywords overlap. Nevertheless, AQL should be easy to understand for anyone with an SQL background.

For some example queries, please refer to the page AQL Examples.

How to invoke AQL

Executing queries

You can run AQL queries from your application via the HTTP REST API. The full API description is available at HTTP Interface for AQL Query Cursors.

You can also run AQL queries from arangosh. To do so, you can use the _query method of the db object. This will run the specified query in the context of the currently selected database and return the query results in a cursor. The results of the cursor can be printed using its toArray method:

arangosh> db._create("mycollection")

arangosh> db.mycollection.save({ _key: "testKey", Hello : "World" })

arangosh> db._query('FOR my IN mycollection RETURN my._key').toArray()

show execution results

To pass bind parameters into a query, they can be specified as second argument to the _query method:

arangosh> db._query(

........> 'FOR c IN @@collection FILTER c._key == @key RETURN c._key', {

........>   '@collection': 'mycollection',

........>   'key': 'testKey'

........> }).toArray();

[

"testKey"

]

It is also possible to use ES6 template strings for generating AQL queries. There is a template string generator function named aqlQuery; we call it once to demonstrate its result, and once putting it directly into the query:

```js

var key = 'testKey';

aqlQuery`FOR c IN mycollection FILTER c._key == ${key} RETURN c._key`;

{

"query" : "FOR c IN mycollection FILTER c._key == @value0 RETURN c._key",

"bindVars" : {

"value0" : "testKey"

}

}

```

arangosh> var key = 'testKey';

arangosh> db._query(

........> aqlQuery`FOR c IN mycollection FILTER c._key == ${key} RETURN c._key`

........> ).toArray();

[

"testKey"

]

Arbitrary JavaScript expressions can be used in queries that are generated with the aqlQuery template string generator. Collection objects are handled automatically:

arangosh> var key = 'testKey';

arangosh> db._query(aqlQuery`FOR doc IN ${ db.mycollection } RETURN doc`

........> ).toArray();

show execution results

Note: data-modification AQL queries normally do not return a result (unless the AQL query contains an extra RETURN statement). When not using a RETURN statement in the query, the toArray method will return an empty array.

It is always possible to retrieve statistics for a query with the getExtra method:

arangosh> db._query(`FOR i IN 1..100

........>             INSERT { _key: CONCAT('test', TO_STRING(i)) }

........>                INTO mycollection`

........> ).getExtra();

show execution results

The meaning of the statistics values is described below.

The _query method is a shorthand for creating an ArangoStatement object, executing it and iterating over the resulting cursor. If more control over the result set iteration is needed, it is recommended to first create an ArangoStatement object as follows:

arangosh> stmt = db._createStatement( {

........> "query": "FOR i IN [ 1, 2 ] RETURN i * 2" } );

[object ArangoStatement]

To execute the query, use the execute method of the statement:

arangosh> c = stmt.execute();

show execution results

This has executed the query. The query results are available in a cursor now. The cursor can return all its results at once using the toArray method. This is a short-cut that you can use if you want to access the full result set without iterating over it yourself.

arangosh> c.toArray();

[

2,

4

]

Cursors can also be used to iterate over the result set document-by-document. To do so, use the hasNext and next methods of the cursor:

arangosh> while (c.hasNext()) { require("internal").print(c.next()); }

2

4

Please note that you can iterate over the results of a cursor only once, and that the cursor will be empty when you have fully iterated over it. To iterate over the results again, the query needs to be re-executed.

Additionally, the iteration can be done in a forward-only fashion. There is no backwards iteration or random access to elements in a cursor.

To execute an AQL query using bind parameters, you need to create a statement first and then bind the parameters to it before execution:

arangosh> var stmt = db._createStatement( {

........> "query": "FOR i IN [ @one, @two ] RETURN i * 2" } );

arangosh> stmt.bind("one", 1);

arangosh> stmt.bind("two", 2);

arangosh> c = stmt.execute();

show execution results

The cursor results can then be dumped or iterated over as usual, e.g.:

arangosh> c.toArray();

[

2,

4

]

or

arangosh> while (c.hasNext()) { require("internal").print(c.next()); }

2

4

Please note that bind parameters can also be passed into the _createStatement method directly, making it a bit more convenient:

arangosh> stmt = db._createStatement( {

........>  "query": "FOR i IN [ @one, @two ] RETURN i * 2",

........>  "bindVars": {

........>    "one": 1,

........>    "two": 2

........>  }

........> } );

[object ArangoStatement]

Cursors also optionally provide the total number of results. By default, they do not. To make the server return the total number of results, you may set the count attribute to true when creating a statement:

arangosh> stmt = db._createStatement( {

........> "query": "FOR i IN [ 1, 2, 3, 4 ] RETURN i",

........> "count": true } );

[object ArangoStatement]

After executing this query, you can use the count method of the cursor to get the number of total results from the result set:

arangosh> var c = stmt.execute();

arangosh> c.count();

4

Please note that the count method returns nothing if you did not specify the count attribute when creating the query.

This is intentional so that the server may apply optimizations when executing the query and construct the result set incrementally. Incremental creation of the result sets is no possible if all of the results need to be shipped to the client anyway. Therefore, the client has the choice to specify count and retrieve the total number of results for a query (and disable potential incremental result set creation on the server), or to not retrieve the total number of results and allow the server to apply optimizations.

Please note that at the moment the server will always create the full result set for each query so specifying or omitting the count attribute currently does not have any impact on query execution. This may change in the future. Future versions of ArangoDB may create result sets incrementally on the server-side and may be able to apply optimizations if a result set is not fully fetched by a client.

Query statistics

A query that has been executed will always return execution statistics. Execution statistics can be retrieved by calling  getExtra()  on the cursor. The statistics are returned in the return value's  stats  attribute:

arangosh> db._query(`

........>   FOR i IN 1..@count INSERT

........>     { _key: CONCAT('anothertest', TO_STRING(i)) }

........>     INTO mycollection`,

........>  {count: 100},

........>  {},

........>  {fullCount: true}

........> ).getExtra();

arangosh> db._query({

........>  "query": `FOR i IN 200..@count INSERT

........>              { _key: CONCAT('anothertest', TO_STRING(i)) }

........>              INTO mycollection`,

........>  "bindVars": {count: 300},

........>  "options": { fullCount: true}

........> }).getExtra();

show execution results

The meaning of the statistics attributes is as follows:

- writesExecuted: the total number of data-modification operations successfully executed. This is equivalent to the number of documents created, updated or removed by  INSERT ,  UPDATE ,  REPLACE  or  REMOVE  operations.

- writesIgnored: the total number of data-modification operations that were unsuccessful, but have been ignored because of query option  ignoreErrors .

- scannedFull: the total number of documents iterated over when scanning a collection without an index. Documents scanned by subqueries will be included in the result, but not no operations triggered by built-in or user-defined AQL functions.

- scannedIndex: the total number of documents iterated over when scanning a collection using an index. Documents scanned by subqueries will be included in the result, but not no operations triggered by built-in or user-defined AQL functions.

- filtered: the total number of documents that were removed after executing a filter condition in a  FilterNode . Note that  IndexRangeNode s can also filter documents by selecting only the required index range from a collection, and the  filtered  value only indicates how much filtering was done by  FilterNode s.

- fullCount: the total number of documents that matched the search condition if the query's final  LIMIT  statement were not present. This attribute will only be returned if the  fullCount  option was set when starting the query and will only contain a sensible value if the query contained a  LIMIT  operation on the top level.

Explaining queries

If it is unclear how a given query will perform, clients can retrieve a query's execution plan from the AQL query optimizer without actually executing the query. Getting the query execution plan from the optimizer is called explaining.

An explain will throw an error if the given query is syntactically invalid. Otherwise, it will return the execution plan and some information about what optimizations could be applied to the query. The query will not be executed.

Explaining a query can be achieved by calling the HTTP REST API. A query can also be explained from the ArangoShell using  ArangoStatement s  explain  method.

By default, the query optimizer will return what it considers to be the optimal plan. The optimal plan will be returned in the  plan  attribute of the result. If  explain  is called with option  allPlans  set to  true , all plans will be returned in the  plans  attribute instead. The result object will also contain an attribute warnings, which is an array of warnings that occurred during optimization or execution plan creation.

Each plan in the result is an object with the following attributes:

- nodes: the array of execution nodes of the plan. The list of available node types can be found here

- estimatedCost: the total estimated cost for the plan. If there are multiple plans, the optimizer will choose the plan with the lowest total cost.

- collections: an array of collections used in the query

- rules: an array of rules the optimizer applied. The list of rules can be found here

- variables: array of variables used in the query (note: this may contain internal variables created by the optimizer)

Here is an example for retrieving the execution plan of a simple query:

arangosh> var stmt = db._createStatement(

........> "FOR user IN _users RETURN user");

arangosh> stmt.explain();

show execution results

As the output of  explain  is very detailed, it is recommended to use some scripting to make the output less verbose:

arangosh> var formatPlan = function (plan) {

........>    return { estimatedCost: plan.estimatedCost,

........>        nodes: plan.nodes.map(function(node) {

........> return node.type; }) }; };

arangosh> formatPlan(stmt.explain().plan);

show execution results

If a query contains bind parameters, they must be added to the statement before  explain  is called:

arangosh> var stmt = db._createStatement(

........> `FOR doc IN @@collection FILTER doc.user == @user RETURN doc`

........> );

arangosh> stmt.bind({ "@collection" : "_users", "user" : "root" });

arangosh> stmt.explain();

show execution results

In some cases the AQL optimizer creates multiple plans for a single query. By default only the plan with the lowest total estimated cost is kept, and the other plans are discarded. To retrieve all plans the optimizer has generated,  explain  can be called with the option  allPlans  set to  true .

In the following example, the optimizer has created two plans:

arangosh> var stmt = db._createStatement(

........> "FOR user IN _users FILTER user.user == 'root' RETURN user");

arangosh> stmt.explain({ allPlans: true }).plans.length;

1

To see a slightly more compact version of the plan, the following transformation can be applied:

arangosh> stmt.explain({ allPlans: true }).plans.map(

........> function(plan) { return formatPlan(plan); });

show execution results

explain  will also accept the following additional options:

- maxPlans: limits the maximum number of plans that are created by the AQL query optimizer

- optimizer.rules: an array of to-be-included or to-be-excluded optimizer rules can be put into this attribute, telling the optimizer to include or exclude specific rules. To disable a rule, prefix its name with a  - , to enable a rule, prefix it with a  + . There is also a pseudo-rule  all , which will match all optimizer rules.

The following example disables all optimizer rules but  remove-redundant-calculations :

arangosh> stmt.explain({ optimizer: {

........> rules: [ "-all", "+remove-redundant-calculations" ] } });

show execution results

The contents of an execution plan are meant to be machine-readable. To get a human-readable version of a query's execution plan, the following commands can be used:

arangosh> var query = "FOR doc IN mycollection FILTER doc.value > 42 RETURN doc";

arangosh> require("org/arangodb/aql/explainer").explain(query, {colors:false});

show execution results

The above command prints the query's execution plan in the ArangoShell directly, focusing on the most important information.

Parsing queries

Clients can use ArangoDB to check if a given AQL query is syntactically valid. ArangoDB provides an HTTP REST API for this.

A query can also be parsed from the ArangoShell using  ArangoStatement s  parse  method. The  parse  method will throw an exception if the query is syntactically invalid. Otherwise, it will return the some information about the query.

The return value is an object with the collection names used in the query listed in the  collections  attribute, and all bind parameters listed in the  bindVars  attribute. Additionally, the internal representation of the query, the query's abstract syntax tree, will be returned in the  AST  attribute of the result. Please note that the abstract syntax tree will be returned without any optimizations applied to it.

arangosh> var stmt = db._createStatement(

........> "FOR doc IN @@collection FILTER doc.foo == @bar RETURN doc");

arangosh> stmt.parse();

show execution results

Data modification queries

AQL supports the following data-modification operations:

- INSERT: insert new documents into a collection

- UPDATE: partially update existing documents in a collection

- REPLACE: completely replace existing documents in a collection

- REMOVE: remove existing documents from a collection

- UPSERT: conditionally insert or update documents in a collection

Modifying a single document

Let's start with the basics:  INSERT ,  UPDATE  and  REMOVE  operations on single documents. Here is an example that insert a document in an existing collection users:

INSERT {firstName:'Anna', name:'Pavlova', profession:'artist'} IN users

You may provide a key for the new document; if not provided, ArangoDB will create one for you.

INSERT {_key:'GilbertoGil', firstName:'Gilberto', name:'Gil', city:'Fortalezza'} IN users

As Arango is schema-free, attributes of the documents may vary:

INSERT {_key:'PhilCarpenter',  firstName:'Phil', name:'Carpenter', middleName:'G.',status:'inactive', } IN users

INSERT {_key:'NatachaDeclerck', firstName:'Natacha', name:'Declerck', location:'Antwerp'} IN users

Update is quite simple. The following AQL statement will add or change the attributes status and location

UPDATE 'PhilCarpenter' WITH { status:'active', location:'Beijing' }  IN users

Replace is an alternative to update where all attributes of the document are replaced.

REPLACE   { _key: 'NatachaDeclerck', firstName:'Natacha', name:'Leclerc', status: 'active', level:'premium' } IN users

Removing a document if you know its key is simple as well :

REMOVE 'GilbertoGil' IN users

or

REMOVE {_key:'GilbertoGil'} IN users

Modifying multiple documents

Data-modification operations are normally combined with FOR loops to iterate over a given list of documents. They can optionally be combined with FILTER statements and the like.

Let's start with an example that modifies existing documents in a collection users that match some condition:

FOR u IN users

FILTER u.status == 'not active'

UPDATE u WITH { status: 'inactive' } IN users

Now, let's copy the contents of the collection users into the collection backup:

FOR u IN users

INSERT u IN backup

As a final example, let's find some documents in collection users and remove them from collection backup. The link between the documents in both collections is established via the documents' keys:

FOR u IN users

FILTER u.status == 'deleted'

REMOVE u IN backup

Returning documents

Data-modification queries can optionally return documents. In order to reference the inserted, removed or modified documents in a  RETURN  statement, data-modification statements introduce the  OLD  and/or  NEW  pseudo-values:

FOR i IN 1..100

INSERT { value: i } IN test

RETURN NEW

FOR u IN users

FILTER u.status == 'deleted'

REMOVE u IN users

RETURN OLD

FOR u IN users

FILTER u.status == 'not active'

UPDATE u WITH { status: 'inactive' } IN users

RETURN NEW

NEW  refers to the inserted or modified document revision, and  OLD  refers to the document revision before update or removal.  INSERT  statements can only refer to the  NEW  pseudo-value, and  REMOVE  operations only to  OLD .  UPDATE ,  REPLACE  and  UPSERT  can refer to either.

In all cases the full documents will be returned with all their attributes, including the potentially auto-generated attributes such as  _id ,  _key , or  _rev  and the attributes not specified in the update expression of a partial update.

Projections

It is possible to return a projection of the documents in  OLD  or  NEW  instead of returning the entire documents. This can be used to reduce the amount of data returned by queries.

For example, the following query will return only the keys of the inserted documents:

FOR i IN 1..100

INSERT { value: i } IN test

RETURN NEW._key

Using OLD and NEW in the same query

For  UPDATE ,  REPLACE  and  UPSERT  statements, both  OLD  and  NEW  can be used to return the previous revision of a document together with the updated revision:

FOR u IN users

FILTER u.status == 'not active'

UPDATE u WITH { status: 'inactive' } IN users

RETURN { old: OLD, new: NEW }

Calculations with OLD or NEW

It is also possible to run additional calculations with  LET  statements between the data-modification part and the final  RETURN  of an AQL query. For example, the following query performs an upsert operation and returns whether an existing document was updated, or a new document was inserted. It does so by checking the  OLD  variable after the  UPSERT  and using a  LET  statement to store a temporary string for the operation type:

UPSERT { name: 'test' }

INSERT { name: 'test' }

UPDATE { } IN users

LET opType = IS_NULL(old) ? 'insert' : 'update'

RETURN { _key: NEW._key, type: opType }

Restrictions

The name of the modified collection (users and backup in the above cases) must be known to the AQL executor at query-compile time and cannot change at runtime. Using a bind parameter to specify the collection name is allowed.

Data-modification queries are restricted to modifying data in a single collection per query. That means a data-modification query cannot modify data in multiple collections with a single query. It is still possible (and was shown above) to read from one or many collections and modify data in another within the same query.

Only a single data-modification operation can be used per AQL query. Data-modification queries cannot be used inside subqueries. Data-modification operations can optionally be followed by  LET  operations and a single  RETURN  operation to return data. If expressions are used within these operations, they cannot contain subqueries or access data in collections using AQL functions.

Finally, data-modification operations can optionally be followed by  LET  and  RETURN , but not by other statements such as  SORT ,  COLLECT  etc.

Transactional Execution

On a single server, data-modification operations are executed transactionally. If a data-modification operation fails, any changes made by it will be rolled back automatically as if they never happened.

In a cluster, AQL data-modification queries are currently not executed transactionally. Additionally, update, replace, upsert and remove AQL queries currently require the _key attribute to be specified for all documents that should be modified or removed, even if a shared key attribute other than _key was chosen for the collection. This restriction may be overcome in a future release of ArangoDB.

The AQL query optimizer

AQL queries are sent through an optimizer before execution. The task of the optimizer is to create an initial execution plan for the query, look for optimization opportunities and apply them. As a result, the optimizer might produce multiple execution plans for a single query. It will then calculate the costs for all plans and pick the plan with the lowest total cost. This resulting plan is considered to be the optimal plan, which is then executed.

The optimizer is designed to only perform optimizations if they are safe, in the meaning that an optimization should not modify the result of a query. A notable exception to this is that the optimizer is allowed to change the order of results for queries that do not explicitly specify how results should be sorted.

Execution plans

The  explain  command can be used to query the optimal executed plan or even all plans the optimizer has generated. Additionally,  explain  can reveal some more information about the optimizer's view of the query.

Inspecting plans using the explain helper

The  explain  method of  ArangoStatement  as shown in the next chapters creates very verbose output. You can work on the output programaticaly, or use this handsome tool that we created to generate a more human readable representation.

You may use it like this: (we disable syntax highlighting here)

arangosh> db._create("test");

arangosh> for (i = 0; i < 100; ++i) { db.test.save({ value: i }); }

arangosh> db.test.ensureIndex({ type: "skiplist", fields: [ "value" ] });

arangosh> var explain = require("org/arangodb/aql/explainer").explain;

arangosh> explain("FOR i IN test FILTER i.value > 97 SORT i.value RETURN i.value", {colors:false});

show execution results

Execution plans in detail

Lets have a look at the raw json output of the same execution plan using the  explain  method of  ArangoStatement :

arangosh> stmt = db._createStatement("FOR i IN test FILTER i.value > 97 SORT i.value RETURN i.value");

arangosh> stmt.explain();

show execution results

The as you can see, result details are very verbose so we will not shown them in full in the next sections. Instead, lets take a closer look at the results step by step.

Execution nodes

In general, an execution plan can be considered to be a pipeline of processing steps. Each processing step is carried out by a so-called execution node

The  nodes  attribute of the  explain  result contains these execution nodes in the execution plan. The output is still very verbose, so here's a shorted form of it:

arangosh> stmt.explain().plan.nodes.map(function (node) { return node.type; });

show execution results

Note that the list of nodes might slightly change in future versions of ArangoDB if new execution node types get added or the optimizer will create somewhat more optimized plans).

When a plan is executed, the query execution engine will start with the node at the bottom of the list (i.e. the ReturnNode).

The ReturnNode's purpose is to return data to the caller. It does not produce data itself, so it will ask the node above itself, this is the CalculationNode in our example. CalculationNodes are responsible for evaluating arbitrary expressions. In our example query, the CalculationNode will evaluate the value of  i.value , which is needed by the ReturnNode. The calculation will be applied for all data the CalculationNode gets from the node above it, in our example the FilterNode.

FilterNodes will only let certain documents pass. Normally, filters are based on the evaluation of an expression. The filters expression result ( i.value > 97 ) is calculated in the CalculationNode above the FilterNode.

Finally, all of this needs to be done for documents of collection  test . This is where the IndexNode enters the game. It will use an index (thus its name) to find certain documents in the collection and ship it down the pipeline in the order required by  SORT i.value . The IndexNode itself has a SingletonNode as its input. The sole purpose of a SingletonNode node is to provide a single empty document as input for other processing steps. It is always the end of the pipeline.

Here's a summary:

- SingletonNode: produces an empty document as input for other processing steps.

- IndexNode: iterates over the index on attribute  value  in collection  test  in the order required by  SORT i.value .

- CalculationNode: evaluates the result of the calculation  i.value > 97  to  true  or  false

- FilterNode: only lets documents pass where above calculation returned  true

- CalculationNode: calculates return value  i.value

- ReturnNode: returns data to the caller

Optimizer rules

Note that in the example, the optimizer has optimized the  SORT  statement away. It can do it safely because there is a sorted skiplist index on  i.value , which it has picked in the IndexNode. As the index values are iterated over in sorted order anyway, the extra SortNode would have been redundant and was removed.

Additionally, the optimizer has done more work to generate an execution plan that avoids as much expensive operations as possible. Here is the list of optimizer rules that were applied to the plan:

arangosh> stmt.explain().plan.rules;

show execution results

Here is the meaning of these rules in context of this query:

- move-calculations-up : moves a CalculationNode as far up in the processing pipeline as possible

- move-filters-up : moves a FilterNode as far up in the processing pipeline as possible

- remove-redundant-calculations : replaces references to variables with references to other variables that contain the exact same result. In the example query,  i.value  is calculated multiple times, but each calculation inside a loop iteration would produce the same value. Therefore, the expression result is shared by several nodes.

- remove-unnecessary-calculations : removes CalculationNodes whose result values are not used in the query. In the example this happens due to the  remove-redundant-calculations  rule having made some calculations unnecessary.

- use-index : use an index to iterate over a collection instead of performing a full collection scan. In the example case this makes sense, as the index can be used for filtering and sorting.

- use-index-for-sort : removes a  SORT  operation if it is already satisfied by traversing over a sorted index

Note that some rules may appear multiple times in the list, with number suffixes. This is due to the same rule being applied multiple times, at different positions in the optimizer pipeline.

Collections used in a query

The list of collections used in a plan (and query) is contained in the  collections  attribute of a plan:

arangosh> stmt.explain().plan.collections

show execution results

The  name  attribute contains the name of the  collection , and  type  is the access type, which can be either  read  or  write .

Variables used in a query

The optimizer will also return a list of variables used in a plan (and query). This list will contain auxiliary variables created by the optimizer itself. This list can be ignored by end users in most cases.

Cost of a query

For each plan the optimizer generates, it will calculate the total cost. The plan with the lowest total cost is considered to be the optimal plan. Costs are estimates only, as the actual execution costs are unknown to the optimizer. Costs are calculated based on heuristics that are hard-coded into execution nodes. Cost values do not have any unit.

Retrieving all execution plans

To retrieve not just the optimal plan but a list of all plans the optimizer has generated, set the option  allPlans  to  true :

This will return a list of all plans in the  plans  attribute instead of in the  plan  attribute:

arangosh> stmt.explain({ allPlans: true });

show execution results

Retrieving the plan as it was generated by the parser / lexer

To retrieve the plan which closely matches your query, you may turn off most optimization rules (i.e. cluster rules cannot be disabled if you're running the explain on a cluster coordinator) set the option  rules  to  -all :

This will return an unoptimized plan in the  plan :

arangosh> stmt.explain({ optimizer: { rules: [ "-all" ] } });

show execution results

Note that some optimizations are already done at parse time (i.e. evaluate simple constant calculation as  1 + 1 )

Turning specific optimizer rules off

Optimizer rules can also be turned on or off individually, using the  rules  attribute. This can be used to enable or disable one or multiple rules. Rules that shall be enabled need to be prefixed with a  + , rules to be disabled should be prefixed with a  - . The pseudo-rule  all  matches all rules.

Rules specified in  rules  are evaluated from left to right, so the following works to turn on just the one specific rule:

arangosh> stmt.explain({ optimizer: { rules: [ "-all", "+use-index-range" ] } });

show execution results

By default, all rules are turned on. To turn off just a few specific rules, use something like this:

arangosh> stmt.explain({ optimizer: { rules: [ "-use-index-range", "-use-index-for-sort" ] } });

show execution results

The maximum number of plans created by the optimizer can also be limited using the  maxNumberOfPlans  attribute:

arangosh> stmt.explain({ maxNumberOfPlans: 1 });

show execution results

Optimizer statistics

The optimizer will return statistics as a part of an  explain  result.

The following attributes will be returned in the  stats  attribute of an  explain  result:

- plansCreated : total number of plans created by the optimizer

- rulesExecuted : number of rules executed (note: an executed rule does not indicate a plan was actually modified by a rule)

- rulesSkipped : number of rules skipped by the optimizer

Warnings

For some queries, the optimizer may produce warnings. These will be returned in the  warnings  attribute of the  explain  result:

arangosh> var stmt = db._createStatement("FOR i IN 1..10 RETURN 1 / 0")

arangosh> stmt.explain().warnings;

show execution results

There is an upper bound on the number of warning a query may produce. If that bound is reached, no further warnings will be returned.

List of execution nodes

The following execution node types will appear in the output of  explain :

- SingletonNode: the purpose of a SingletonNode is to produce an empty document that is used as input for other processing steps. Each execution plan will contain exactly one SingletonNode as its top node.

- EnumerateCollectionNode: enumeration over documents of a collection (given in its collection attribute) without using an index.

- IndexNode: enumeration over one or many indexes (given in its indexes attribute) of a collection. The index ranges are specified in the condition attribute of the node.

- EnumerateListNode: enumeration over a list of (non-collection) values.

- FilterNode: only lets values pass that satisfy a filter condition. Will appear once per FILTER statement.

- LimitNode: limits the number of results passed to other processing steps. Will appear once per LIMIT statement.

- CalculationNode: evaluates an expression. The expression result may be used by other nodes, e.g. FilterNode, EnumerateListNode, SortNode etc.

- SubqueryNode: executes a subquery.

- SortNode: performs a sort of its input values.

- AggregateNode: aggregates its input and produces new output variables. This will appear once per COLLECT statement.

- ReturnNode: returns data to the caller. Will appear in each read-only query at least once. Subqueries will also contain ReturnNodes.

- InsertNode: inserts documents into a collection (given in its collection attribute). Will appear exactly once in a query that contains an INSERT statement.

- RemoveNode: removes documents from a collection (given in its collection attribute). Will appear exactly once in a query that contains a REMOVE statement.

- ReplaceNode: replaces documents in a collection (given in its collection attribute). Will appear exactly once in a query that contains a REPLACE statement.

- UpdateNode: updates documents in a collection (given in its collection attribute). Will appear exactly once in a query that contains an UPDATE statement.

- UpsertNode: upserts documents in a collection (given in its collection attribute). Will appear exactly once in a query that contains an UPSERT statement.

- NoResultsNode: will be inserted if FILTER statements turn out to be never satisfiable. The NoResultsNode will pass an empty result set into the processing pipeline.

For queries in the cluster, the following nodes may appear in execution plans:

- ScatterNode: used on a coordinator to fan-out data to one or multiple shards.

- GatherNode: used on a coordinator to aggregate results from one or many shards into a combined stream of results.

- DistributeNode: used on a coordinator to fan-out data to one or multiple shards, taking into account a collection's shard key.

- RemoteNode: a RemoteNode will perform communication with another ArangoDB instances in the cluster. For example, the cluster coordinator will need to communicate with other servers to fetch the actual data from the shards. It will do so via RemoteNodes. The data servers themselves might again pull further data from the coordinator, and thus might also employ RemoteNodes. So, all of the above cluster relevant nodes will be accompanied by a RemoteNode.

List of optimizer rules

The following optimizer rules may appear in the  rules  attribute of a plan:

- move-calculations-up : will appear if a CalculationNode was moved up in a plan. The intention of this rule is to move calculations up in the processing pipeline as far as possible (ideally out of enumerations) so they are not executed in loops if not required. It is also quite common that this rule enables further optimizations to kick in.

- move-filters-up : will appear if a FilterNode was moved up in a plan. The intention of this rule is to move filters up in the processing pipeline as far as possible (ideally out of inner loops) so they filter results as early as possible.

- sort-in-values : will appear when the values used as right-hand side of an  IN  operator will be pre-sorted using an extra function call. Pre-sorting the comparison array allows using a binary search in-list lookup with a logarithmic complexity instead of the default linear complexity in-list lookup.

- remove-unnecessary-filters : will appear if a FilterNode was removed or replaced. FilterNodes whose filter condition will always evaluate to true will be removed from the plan, whereas FilterNode that will never let any results pass will be replaced with a NoResultsNode.

- remove-redundant-calculations : will appear if redundant calculations (expressions with the exact same result) were found in the query. The optimizer rule will then replace references to the redundant expressions with a single reference, allowing other optimizer rules to remove the then-unneeded CalculationNodes.

- remove-unnecessary-calculations : will appear if CalculationNodes were removed from the query. The rule will removed all calculations whose result is not referenced in the query (note that this may be a consequence of applying other optimizations).

- remove-redundant-sorts : will appear if multiple SORT statements can be merged into fewer sorts.

- interchange-adjacent-enumerations : will appear if a query contains multiple FOR statements whose order were permuted. Permutation of FOR statements is performed because it may enable further optimizations by other rules.

- remove-sort-rand : will appear when a SORT RAND() expression is removed by moving the random iteration into an EnumerateCollectionNode.

- remove-collect-variables : will appear if an INTO clause was removed from a COLLECT statement because the result of INTO is not used. May also appear if a result of a COLLECT statement's AGGREGATE variables is not used.

- propagate-constant-attributes : will appear when a constant value was inserted into a filter condition, replacing a dynamic attribute value.

- replace-or-with-in : will appear if multiple OR-combined equality conditions on the same variable or attribute were replaced with an IN condition.

- remove-redundant-or : will appear if multiple OR conditions for the same variable or attribute were combined into a single condition.

- use-indexes : will appear when an index is used to iterate over a collection. As a consequence, an EnumerateCollectionNode was replaced with an IndexNode in the plan.

- remove-filters-covered-by-index : will appear if a FilterNode was removed or replaced because the filter condition is already covered by an IndexNode.

- use-index-for-sort : will appear if an index can be used to avoid a SORT operation. If the rule was applied, a SortNode was removed from the plan.

- move-calculations-down : will appear if a CalculationNode was moved down in a plan. The intention of this rule is to move calculations down in the processing pipeline as far as possible (below FILTER, LIMIT and SUBQUERY nodes) so they are executed as late as possible and not before their results are required.

- patch-update-statements : will appear if an UpdateNode was patched to not buffer its input completely, but to process it in smaller batches. The rule will fire for an UPDATE query that is fed by a full collection scan, and that does not use any other indexes and subqueries.

The following optimizer rules may appear in the  rules  attribute of cluster plans:

- distribute-in-cluster : will appear when query parts get distributed in a cluster. This is not an optimization rule, and it cannot be turned off.

- scatter-in-cluster : will appear when scatter, gather, and remote nodes are inserted into a distributed query. This is not an optimization rule, and it cannot be turned off.

- distribute-filtercalc-to-cluster : will appear when filters are moved up in a distributed execution plan. Filters are moved as far up in the plan as possible to make result sets as small as possible as early as possible.

- distribute-sort-to-cluster : will appear if sorts are moved up in a distributed query. Sorts are moved as far up in the plan as possible to make result sets as small as possible as early as possible.

- remove-unnecessary-remote-scatter : will appear if a RemoteNode is followed by a ScatterNode, and the ScatterNode is only followed by calculations or the SingletonNode. In this case, there is no need to distribute the calculation, and it will be handled centrally.

- undistribute-remove-after-enum-coll : will appear if a RemoveNode can be pushed into the same query part that enumerates over the documents of a collection. This saves inter-cluster roundtrips between the EnumerateCollectionNode and the RemoveNode.

Note that some rules may appear multiple times in the list, with number suffixes. This is due to the same rule being applied multiple times, at different positions in the optimizer pipeline.

The AQL query result cache

AQL provides an optional query result cache.

The purpose of the query cache is to avoid repeated calculation of the same query results. It is useful if data-reading queries repeat a lot and there are not many write queries.

The query cache is transparent so users do not need to manually invalidate results in it if underlying collection data are modified.

Modes

The cache can be operated in the following modes:

- off : the cache is disabled. No query results will be stored

- on : the cache will store the results of all AQL queries unless their  cache  attribute flag is set to  false

- demand : the cache will store the results of AQL queries that have their  cache  attribute set to  true , but will ignore all others

The mode can be set at server startup and later changed at runtime.

Query eligibility

The query cache will consider two queries identical if they have exactly the same query string. Any deviation in terms of whitespace, capitalization etc. will be considered a difference. The query string will be hashed and used as the cache lookup key. If a query uses bind parameters, these will also be hashed and used as the cache lookup key.

That means even if the query string for two queries is identical, the query cache will treat them as different queries if they have different bind parameter values. Other components that will become part of a query's cache key are the  count  and  fullCount  attributes.

If the cache is turned on, the cache will check at the very start of execution whether it has a result ready for this particular query. If that is the case, the query result will be served directly from the cache, which is normally very efficient. If the query cannot be found in the cache, it will be executed as usual.

If the query is eligible for caching and the cache is turned on, the query result will be stored in the query cache so it can be used for subsequent executions of the same query.

A query is eligible for caching only if all of the following conditions are met:

- the server the query executes on is not a coordinator

- the query string is at least 8 characters long

- the query is a read-only query and does not modify data in any collection

- no warnings were produced while executing the query

- the query is deterministic and only uses deterministic functions

The usage of non-deterministic functions leads to a query not being cachable. This is intentional to avoid caching of function results which should rather be calculated on each invocation of the query (e.g.  RAND()  or  DATE_NOW() ).

The query cache considers all user-defined AQL functions to be non-deterministic as it has no insight into these functions.

Cache invalidation

The query cache results are fully or partially invalidated automatically if queries modify the data of collections that were used during the computation of the cached query results. This is to protect users from getting stale results from the query cache.

This also means that if the cache is turned on, then there is an additional cache invalidation check for each data-modification operation (e.g. insert, update, remove, truncate operations as well as AQL data-modification queries).

Example

If the result of the following query is present in the query cache, then either modifying data in collection  users  or in collection  organizations  will remove the already computed result from the cache:

FOR user IN users

FOR organization IN organizations

FILTER user.organization == organization._key

RETURN { user: user, organization: organization }

Modifying data in other collections than the named two will not lead to this query result being removed from the cache.

Performance considerations

The query cache is organized as a hash table, so looking up whether a query result is present in the cache is relatively fast. Still, the query string and the bind parameter used in the query will need to be hashed. This is a slight overhead that will not be present if the cache is turned off or a query is marked as not cacheable.

Additionally, storing query results in the cache and fetching results from the cache requires locking via an R/W lock. While many thread can read in parallel from the cache, there can only be a single modifying thread at any given time. Modifications of the query cache contents are required when a query result is stored in the cache or during cache invalidation after data-modification operations. Cache invalidation will require time proportional to the number of cached items that need to be invalidated.

There may be workloads in which enabling the query cache will lead to a performance degradation. It is not recommended to turn the query cache on in workloads that only modify data, or that modify data more often than read it. Turning on the query cache will also provide no benefit if queries are very diverse and do not repeat often. In read-only or read-mostly workloads, the query cache will be beneficial if the same queries are repeated lots of times.

In general, the query cache will provide the biggest improvements for queries with small result sets that take long to calculate. If a query's results are very big and most of the query time is spent in copying the result from the cache to the client, then the cache will not provide much benefit.

Global configuration

The query cache can be configured at server start using the configuration parameter  --database.query-cache-mode . This will set the cache mode according to the descriptions above.

After the server is started, the cache mode can be changed at runtime as follows:

require("org/arangodb/aql/cache").properties({ mode: "on" });

The maximum number of cached results in the cache for each database can be configured at server start using the configuration parameter  --database.query-cache-max-results . This parameter can be used to put an upper bound on the number of query results in each database's query cache and thus restrict the cache's memory consumption.

The value can also be adjusted at runtime as follows:

require("org/arangodb/aql/cache").properties({ maxResults: 200 });

Per-query configuration

When a query is sent to the server for execution and the cache is set to  on  or  demand , the query executor will look into the query's  cache  attribute. If the query cache mode is  on , then not setting this attribute or setting it to anything but  false  will make the query executor consult the query cache. If the query cache mode is  demand , then setting the  cache  attribute to  true  will make the executor look for the query in the query cache. When the query cache mode is  off , the executor will not look for the query in the cache.

The  cache  attribute can be set as follows via the  db._createStatement()  function:

var stmt = db._createStatement({

query: "FOR doc IN users LIMIT 5 RETURN doc",

cache: true  /* cache attribute set here */

});

stmt.execute();

When using the  db._query()  function, the  cache  attribute can be set as allows:

db._query({

query: "FOR doc IN users LIMIT 5 RETURN doc",

cache: true  /* cache attribute set here */

});

The  cache  attribute can be set via the HTTP REST API  POST /_api/cursor , too.

Each query result returned will contain a  cached  attribute. This will be set to  true  if the result was retrieved from the query cache, and  false  otherwise. Clients can use this attribute to check if a specific query was served from the cache or not.

Restrictions

Query results that are returned from the query cache do not contain any execution statistics, meaning their extra.stats attribute will not be present. Additionally query results returned from the cache will not contain profile information even if the profile option was set to true when invoking the query.

Language basics

Query types

An AQL query must either return a result (indicated by usage of the RETURN keyword) or execute a data-modification operation (indicated by usage of one of the keywords INSERT, UPDATE, REPLACE, REMOVE or UPSERT). The AQL parser will return an error if it detects more than one data-modification operation in the same query or if it cannot figure out if the query is meant to be a data retrieval or a modification operation.

AQL only allows one query in a single query string; thus semicolons to indicate the end of one query and separate multiple queries (as seen in SQL) are not allowed.

Whitespace

Whitespaces (blanks, carriage returns, line feeds, and tab stops) can be used in the query text to increase its readability. Tokens have to be separated by any number of whitespaces. Whitespace within strings or names must be enclosed in quotes in order to be preserved.

Comments

Comments can be embedded at any position in a query. The text contained in the comment is ignored by the AQL parser.

Multi-line comments cannot be nested, which means subsequent comment starts within comments are ignored, comment ends will end the comment.

AQL supports two types of comments:

- Single line comments: These start with a double forward slash and end at the end of the line, or the end of the query string (whichever is first).

- Multi line comments: These start with a forward slash and asterisk, and end with an asterisk and a following forward slash. They can span as many lines as necessary.

/* this is a comment */ RETURN 1

/* these */ RETURN /* are */ 1 /* multiple */ + /* comments */ 1

/* this is

a multi line

comment */

// a single line comment

Keywords

On the top level, AQL offers the following operations:

- FOR : array iteration

- RETURN : results projection

- FILTER : results filtering

- SORT : result sorting

- LIMIT : result slicing

- LET : variable assignment

- COLLECT : result grouping

- INSERT : insertion of new documents

- UPDATE : (partial) update of existing documents

- REPLACE : replacement of existing documents

- REMOVE : removal of existing documents

- UPSERT : insertion or update of existing documents

Each of the above operations can be initiated in a query by using a keyword of the same name. An AQL query can (and typically does) consist of multiple of the above operations.

An example AQL query may look like this:

FOR u IN users

FILTER u.type == "newbie" && u.active == true

RETURN u.name

In this example query, the terms FOR, FILTER, and RETURN initiate the higher-level operation according to their name. These terms are also keywords, meaning that they have a special meaning in the language.

For example, the query parser will use the keywords to find out which high-level operations to execute. That also means keywords can only be used at certain locations in a query. This also makes all keywords reserved words that must not be used for other purposes than they are intended for.

For example, it is not possible to use a keyword as a collection or attribute name. If a collection or attribute need to have the same name as a keyword, the collection or attribute name needs to be quoted.

Keywords are case-insensitive, meaning they can be specified in lower, upper, or mixed case in queries. In this documentation, all keywords are written in upper case to make them distinguishable from other query parts.

In addition to the higher-level operations keywords, there are other keywords. The current list of keywords is:

- FOR

- RETURN

- FILTER

- SORT

- LIMIT

- LET

- COLLECT

- INSERT

- UPDATE

- REPLACE

- REMOVE

- UPSERT

- WITH

- ASC

- DESC

- IN

- INTO

- NOT

- AND

- OR

- NULL

- TRUE

- FALSE

- DISTINCT

- GRAPH

- OUTBOUND

- INBOUND

- ANY

- ALL

- NONE

- AGGREGATE

Additional keywords may be added in future versions of ArangoDB.

Names

In general, names are used to identify objects (collections, attributes, variables, and functions) in AQL queries.

The maximum supported length of any name is 64 bytes. Names in AQL are always case-sensitive.

Keywords must not be used as names. If a reserved keyword should be used as a name, the name must be enclosed in backticks. Enclosing a name in backticks makes it possible to use otherwise reserved keywords as names. An example for this is:

FOR f IN `filter`

RETURN f.`sort`

Due to the backticks, filter and sort are interpreted as names and not as keywords here.

Collection names

Collection names can be used in queries as they are. If a collection happens to have the same name as a keyword, the name must be enclosed in backticks.

Please refer to the Naming Conventions in ArangoDB about collection naming conventions.

Attribute names

When referring to attributes of documents from a collection, the fully qualified attribute name must be used. This is because multiple collections with ambiguous attribute names may be used in a query. To avoid any ambiguity, it is not allowed to refer to an unqualified attribute name.

Please refer to the Naming Conventions in ArangoDB for more information about the attribute naming conventions.

FOR u IN users

FOR f IN friends

FILTER u.active == true && f.active == true && u.id == f.userId

RETURN u.name

In the above example, the attribute names active, name, id, and userId are qualified using the collection names they belong to (u and f respectively).

Variable names

AQL allows the user to assign values to additional variables in a query. All variables that are assigned a value must have a name that is unique within the context of the query. Variable names must be different from the names of any collection name used in the same query.

FOR u IN users

LET friends = u.friends

RETURN { "name" : u.name, "friends" : friends }

In the above query, users is a collection name, and both u and friends are variable names. This is because the FOR and LET operations need target variables to store their intermediate results.

Allowed characters in variable names are the letters a to z (both in lower and upper case), the numbers 0 to 9 and the underscore (_) symbol. A variable name must not start with a number. If a variable name starts with the underscore character, it must also contain at least one letter (a-z or A-Z).

Data types

AQL supports both primitive and compound data types. The following types are available:

- Primitive types: Consisting of exactly one value

- null: An empty value, also: The absence of a value

- bool: Boolean truth value with possible values false and true

- number: Signed (real) number

- string: UTF-8 encoded text value

- Compound types: Consisting of multiple values

- array: Sequence of values, referred to by their positions

- object / document: Sequence of values, referred to by their names

Numeric literals

Numeric literals can be integers or real values. They can optionally be signed using the + or - symbols. The scientific notation is also supported.

1

42

-1

-42

1.23

-99.99

0.1

-4.87e103

All numeric values are treated as 64-bit double-precision values internally. The internal format used is IEEE 754.

String literals

String literals must be enclosed in single or double quotes. If the used quote character is to be used itself within the string literal, it must be escaped using the backslash symbol. Backslash literals themselves also be escaped using a backslash.

"yikes!"

"don't know"

"this is a \"quoted\" word"

"this is a longer string."

"the path separator on Windows is \\"

'yikes!'

'don\'t know'

'this is a longer string."

'the path separator on Windows is \\'

All string literals must be UTF-8 encoded. It is currently not possible to use arbitrary binary data if it is not UTF-8 encoded. A workaround to use binary data is to encode the data using base64 or other algorithms on the application side before storing, and decoding it on application side after retrieval.

Arrays

AQL supports two compound types:

- arrays: A composition of unnamed values, each accessible by their positions

- objects / documents: A composition of named values, each accessible by their names

The first supported compound type is the array type. Arrays are effectively sequences of (unnamed / anonymous) values. Individual array elements can be accessed by their positions. The order of elements in an array is important.

An array-declaration starts with the [ symbol and ends with the ] symbol. An array-declaration contains zero or many expressions, separated from each other with the , symbol.

In the easiest case, an array is empty and thus looks like:

[ ]

Array elements can be any legal expression values. Nesting of arrays is supported.

[ 1, 2, 3 ]

[ -99, "yikes!", [ true, [ "no"], [ ] ], 1 ]

[ [ "fox", "marshal" ] ]

Individual array values can later be accesses bd their positions using the [] accessor. The position of the accessed element must be a numeric value. Positions start at 0. It is also possible to use negative index values to access array values starting from the end of the array. This is convenient if the length of the array is unknown and access to elements at the end of the array is required.

// access 1st array element (element start at index 0)

u.friends[0]

// access 3rd array element

u.friends[2]

// access last array element

u.friends[-1]

// access second last array element

u.friends[-2]

Objects / Documents

The other supported compound type is the object (or document) type. Objects are a composition of zero to many attributes. Each attribute is a name/value pair. Object attributes can be accessed individually by their names.

Object declarations start with the { symbol and end with the } symbol. An object contains zero to many attribute declarations, separated from each other with the , symbol. In the simplest case, an object is empty. Its declaration would then be:

{ }

Each attribute in an object is a name / value pair. Name and value of an attribute are separated using the : symbol.

The attribute name is mandatory and must be specified as a quoted or unquoted string. If a keyword is used as an attribute name, the attribute name must be quoted:

{ return : 1 }     /* won't work */

{ "return" : 1 }   /* works ! */

{ `return` : 1 }   /* works, too! */

Since ArangoDB 2.6, object attribute names can be computed using dynamic expressions, too. To disambiguate regular attribute names from attribute name expressions, computed attribute names must be enclosed in [ and ]:

{ [ CONCAT("test/", "bar") ] : "someValue" }

Since ArangoDB 2.7, there is also shorthand notation for attributes which is handy for returning existing variables easily:

LET name = "Peter"

LET age = 42

RETURN { name, age }

The above is the shorthand equivalent for the generic form:

LET name = "Peter"

LET age = 42

RETURN { name : name, age : age }

Any valid expression can be used as an attribute value. That also means nested objects can be used as attribute values:

{ name : "Peter" }

{ "name" : "Vanessa", "age" : 15 }

{ "name" : "John", likes : [ "Swimming", "Skiing" ], "address" : { "street" : "Cucumber lane", "zip" : "94242" } }

Individual object attributes can later be accessed by their names using the . accessor. If a non-existing attribute is accessed, the result is null.

u.address.city.name

u.friends[0].name.first

Bind parameters

AQL supports the usage of bind parameters, thus allowing to separate the query text from literal values used in the query. It is good practice to separate the query text from the literal values because this will prevent (malicious) injection of keywords and other collection names into an existing query. This injection would be dangerous because it may change the meaning of an existing query.

Using bind parameters, the meaning of an existing query cannot be changed. Bind parameters can be used everywhere in a query where literals can be used.

The syntax for bind parameters is @name where name is the actual parameter name. The bind parameter values need to be passed along with the query when it is executed, but not as part of the query text itself.

FOR u IN users

FILTER u.id == @id && u.name == @name

RETURN u

Bind parameter names must start with any of the letters a to z (both in lower and upper case) or a digit (0 to 9), and can be followed by any letter, digit or the underscore symbol.

Bind variables represent a value like a string, and must not be put in quotes. If you need to do string processing (concatenation, etc.) in the AQL query, you need to use string functions to do so:

FOR u IN users

FILTER u.id == CONCAT('prefix', @id, 'suffix') && u.name == @name

RETURN u

A special type of bind parameter exists for injecting collection names. This type of bind parameter has a name prefixed with an additional @ symbol (thus when using the bind parameter in a query, two @ symbols must be used).

FOR u IN @@collection

FILTER u.active == true

RETURN u

Type and value order

When checking for equality or inequality or when determining the sort order of values, AQL uses a deterministic algorithm that takes both the data types and the actual values into account.

The compared operands are first compared by their data types, and only by their data values if the operands have the same data types.

The following type order is used when comparing data types:

null < bool < number < string < array < object / document

This means null is the smallest type in AQL and document is the type with the highest order. If the compared operands have a different type, then the comparison result is determined and the comparison is finished.

For example, the boolean true value will always be less than any numeric or string value, any array (even an empty array) or any object / document. Additionally, any string value (even an empty string) will always be greater than any numeric value, a boolean value, true or false.

null < false

null < true

null < 0

null < ''

null < ' '

null < '0'

null < 'abc'

null < [ ]

null < { }

false < true

false < 0

false < ''

false < ' '

false < '0'

false < 'abc'

false < [ ]

false < { }

true < 0

true < ''

true < ' '

true < '0'

true < 'abc'

true < [ ]

true < { }

0 < ''

0 < ' '

0 < '0'

0 < 'abc'

0 < [ ]

0 < { }

'' < ' '

'' < '0'

'' < 'abc'

'' < [ ]

'' < { }

[ ] < { }

If the two compared operands have the same data types, then the operands values are compared. For the primitive types (null, boolean, number, and string), the result is defined as follows:

- null: null is equal to null

- boolean: false is less than true

- number: numeric values are ordered by their cardinal value

- string: string values are ordered using a localized comparison,

Note: unlike in SQL, null can be compared to any value, including null itself, without the result being converted into null automatically.

For compound, types the following special rules are applied:

Two array values are compared by comparing their individual elements position by position, starting at the first element. For each position, the element types are compared first. If the types are not equal, the comparison result is determined, and the comparison is finished. If the types are equal, then the values of the two elements are compared. If one of the arrays is finished and the other array still has an element at a compared position, then null will be used as the element value of the fully traversed array.

If an array element is itself a compound value (an array or an object / document), then the comparison algorithm will check the element's sub values recursively. The element's sub-elements are compared recursively.

[ ] < [ 0 ]

[ 1 ] < [ 2 ]

[ 1, 2 ] < [ 2 ]

[ 99, 99 ] < [ 100 ]

[ false ] < [ true ]

[ false, 1 ] < [ false, '' ]

Two object / documents operands are compared by checking attribute names and value. The attribute names are compared first. Before attribute names are compared, a combined array of all attribute names from both operands is created and sorted lexicographically. This means that the order in which attributes are declared in an object / document is not relevant when comparing two objects / documents.

The combined and sorted array of attribute names is then traversed, and the respective attributes from the two compared operands are then looked up. If one of the objects / documents does not have an attribute with the sought name, its attribute value is considered to be null. Finally, the attribute value of both objects / documents is compared using the before mentioned data type and value comparison. The comparisons are performed for all object / document attributes until there is an unambiguous comparison result. If an unambiguous comparison result is found, the comparison is finished. If there is no unambiguous comparison result, the two compared objects / documents are considered equal.

{ } < { "a" : 1 }

{ } < { "a" : null }

{ "a" : 1 } < { "a" : 2 }

{ "b" : 1 } < { "a" : 0 }

{ "a" : { "c" : true } } < { "a" : { "c" : 0 } }

{ "a" : { "c" : true, "a" : 0 } } < { "a" : { "c" : false, "a" : 1 } }

{ "a" : 1, "b" : 2 } == { "b" : 2, "a" : 1 }

Accessing data from collections

Collection data can be accessed by specifying a collection name in a query. A collection can be understood as an array of documents, and that is how they are treated in AQL. Documents from collections are normally accessing using the FOR keyword. Note that when iterating over documents from a collection, the order of documents is undefined. To traverse documents in an explicit and deterministic order, the SORT keyword should be used in addition.

Data in collections is stored in documents, with each document potentially having different attributes than other documents. This is true even for documents of the same collection.

It is therefore quite normal to encounter documents that do not have some or all of the attributes that are queried in an AQL query. In this case, the non-existing attributes in the document will be treated as if they would exist with a value of null. That means that comparing a document attribute to null will return true if the document has the particular attribute and the attribute has a value of null, or that the document does not have the particular attribute at all.

For example, the following query will return all documents from the collection users that have a value of null in the attribute name, plus all documents from users that do not have the name attribute at all:

FOR u IN users

FILTER u.name == null

RETURN u

Furthermore, null is less than any other value (excluding null itself). That means documents with non-existing attributes may be included in the result when comparing attribute values with the less than or less equal operators.

For example, the following query will return all documents from the collection users that have an attribute age with a value less than 39, but also all documents from the collection that do not have the attribute age at all.

FOR u IN users

FILTER u.age < 39

RETURN u

This behavior should always be taken into account when writing queries.

Functions

AQL supports functions to allow more complex computations. Functions can be called at any query position where an expression is allowed. The general function call syntax is:

FUNCTIONNAME(arguments)

where FUNCTIONNAME is the name of the function to be called, and arguments is a comma-separated list of function arguments. If a function does not need any arguments, the argument list can be left empty. However, even if the argument list is empty the parentheses around it are still mandatory to make function calls distinguishable from variable names.

Some example function calls:

HAS(user, "name")

LENGTH(friends)

COLLECTIONS()

In contrast to collection and variable names, function names are case-insensitive, i.e. LENGTH(foo) and length(foo) are equivalent.

Extending AQL

Since ArangoDB 1.3, it is possible to extend AQL with user-defined functions. These functions need to be written in JavaScript, and be registered before usage in a query.

Please refer to Extending AQL for more details on this.

By default, any function used in an AQL query will be sought in the built-in function namespace _aql. This is the default namespace that contains all AQL functions that are shipped with ArangoDB. To refer to a user-defined AQL function, the function name must be fully qualified to also include the user-defined namespace. The :: symbol is used as the namespace separator:

MYGROUP::MYFUNC()

MYFUNCTIONS::MATH::RANDOM()

As all AQL function names, user function names are also case-insensitive.

Type cast functions

As mentioned before, some of the operators expect their operands to have a certain data type. For example, the logical operators expect their operands to be boolean values, and the arithmetic operators expect their operands to be numeric values. If an operation is performed with operands of an unexpected type, the operation will fail with an error. To avoid such failures, value types can be converted explicitly in a query. This is called type casting.

In an AQL query, type casts are performed only upon request and not implicitly. This helps avoiding unexpected results. All type casts have to be performed by invoking a type cast function. AQL offers several type cast functions for this task. Each of the these functions takes an operand of any data type and returns a result value of type corresponding to the function name (e.g. TO_NUMBER() will return a number value):

- TO_BOOL(value): Takes an input value of any type and converts it into the appropriate boolean value as follows:

- null is converted to false.

- Numbers are converted to true if they are unequal to 0, and to false otherwise.

- Strings are converted to true if they are non-empty, and to false otherwise.

- Arrays are always converted to true.

- Objects / documents are always converted to true.

- TO_NUMBER(value): Takes an input value of any type and converts it into a numeric value as follows:

- null and false are converted to the value 0.

- true is converted to 1.

- Numbers keep their original value.

- Strings are converted to their numeric equivalent if the string contains a valid representation of a number. Whitespace at the start and end of the string is allowed. String values that do not contain any valid representation of a number will be converted to null.

- An empty array is converted to 0, an array with one member is converted into the result of  TO_NUMBER()  for its sole member. An array with two or more members is converted to null.

- An object / document is converted to null.

- TO_STRING(value): Takes an input value of any type and converts it into a string value as follows:

- null is converted to the string "null"

- false is converted to the string "false", true to the string "true"

- Numbers are converted to their string representations.

- An empty array is converted to the empty string. An array with one member is converted to the result of  TO_STRING()  for its sole member. An array with two or more members is converted to a comma-separated array with the string representation of its members.

- An object / document is converted to the string [object Object].

- TO_ARRAY(value): Takes an input value of any type and converts it into an array value as follows:

- null is converted to an empty array.

- Boolean values, numbers and strings are converted to an array containing the original value as its single element.

- Arrays keep their original value.

- Objects / documents are converted to an array containing their attribute values as array elements

- TO_LIST(value): This is an alias for TO_ARRAY.

Type check functions

AQL also offers functions to check the data type of a value at runtime. The following type check functions are available. Each of these functions takes an argument of any data type and returns true if the value has the type that is checked for, and false otherwise.

The following type check functions are available:

- IS_NULL(value): Checks whether value is a null value

- IS_BOOL(value): Checks whether value is a boolean value

- IS_NUMBER(value): Checks whether value is a numeric value

- IS_STRING(value): Checks whether value is a string value

- IS_ARRAY(value): Checks whether value is an array value

- IS_LIST(value): This is an alias for IS_ARRAY

- IS_OBJECT(value): Checks whether value is an object / document value

- IS_DOCUMENT(value): This is an alias for IS_OBJECT

- IS_DATESTRING(value): Checks whether value is a string that can be used in a date function. This includes partial dates such as 2015 or 2015-10 and strings containing invalid dates such as 2015-02-31. The function will return false for all non-string values, even if some of them may be usable in date functions.

String functions

For string processing, AQL offers the following functions:

- CONCAT(value1, value2, ... valueN): Concatenate the strings passed as in value1 to valueN. null values are ignored. Array value arguments are expanded automatically, and their individual members will be concatenated.

/* "foobarbaz" */

CONCAT('foo', 'bar', 'baz')

/* "foobarbaz" */

CONCAT([ 'foo', 'bar', 'baz' ])

- CONCAT_SEPARATOR(separator, value1, value2, ... valueN): Concatenate the strings passed as arguments value1 to valueN using the separator string. null values are ignored. Array value arguments are expanded automatically, and their individual members will be concatenated.

/* "foo, bar, baz" */

CONCAT_SEPARATOR(', ', 'foo', 'bar', 'baz')

/* "foo, bar, baz" */

CONCAT_SEPARATOR(', ', [ 'foo', 'bar', 'baz' ])

- CHAR_LENGTH(value): Return the number of characters in value. This is a synonym for LENGTH(value)

- LOWER(value): Lower-case value

- UPPER(value): Upper-case value

- SUBSTITUTE(value, search, replace, limit): Replaces search values in the string value. If search is a string, all occurrences of search will be replaced in value. If search is a list, each occurrence of a value contained in search will be replaced by the corresponding list item in replace. If replace has less list items than search, occurrences of unmapped search items will be replaced by the empty string. The number of replacements can optionally be limited using the limit parameter. If the limit is reached, no further occurrences of the search values will be replaced.

/* "the lazy brown foxx" */

SUBSTITUTE("the quick brown foxx", "quick", "lazy")

/* "the slow brown dog" */

SUBSTITUTE("the quick brown foxx", [ "quick", "foxx" ], [ "slow", "dog" ])

/* "A VOID! brown " */

SUBSTITUTE("the quick brown foxx", [ "the", "quick", "foxx" ], [ "A", "VOID!" ])

/* "the xx brown xx" */

SUBSTITUTE("the quick brown foxx", [ "quick", "foxx" ], "xx" )

Alternatively, search and replace can be specified in a combined value:

/* "the small slow ant" */

SUBSTITUTE("the quick brown foxx", {

"quick" : "small",

"brown" : "slow",

"foxx" : "ant"

})

- SUBSTRING(value, offset, length): Return a substring of value, starting at offset and with a maximum length of length characters. Offsets start at position 0. Length is optional and if omitted the substring from offset to the end of the string will be returned.

- LEFT(value, LENGTH): Returns the LENGTH leftmost characters of the string value

- RIGHT(value, LENGTH): Returns the LENGTH rightmost characters of the string value

- TRIM(value, type): Returns the string value with whitespace stripped from the start and/or end. The optional type parameter specifies from which parts of the string the whitespace is stripped:

- type 0 will strip whitespace from the start and end of the string

- type 1 will strip whitespace from the start of the string only

- type 2 will strip whitespace from the end of the string only

- TRIM(value, chars): Returns the string value with whitespace stripped from the start and end. The optional chars parameter can be used to override the characters that should be removed from the string. It defaults to  \r\n\t  (i.e.  0x0d ,  0x0a ,  0x09  and  0x20 ).

/* "foobar" */

TRIM("  foobar\t \r\n ")

/* "foo;bar;baz" */

TRIM(";foo;bar;baz, ", ",; ")

- LTRIM(value, chars): Returns the string value with whitespace stripped from the start only. The optional chars parameter can be used to override the characters that should be removed from the string. It defaults to  \r\n\t  (i.e.  0x0d ,  0x0a ,  0x09  and  0x20 ).

- RTRIM(value, chars): Returns the string value with whitespace stripped from the end only. The optional chars parameter can be used to override the characters that should be removed from the string. It defaults to  \r\n\t  (i.e.  0x0d ,  0x0a ,  0x09  and  0x20 ).

- SPLIT(value, separator, limit): Splits the given string value into a list of strings, using the separator. The separator can either be a string or a list of strings. If the separator is the empty string, value will be split into a list of characters. If no separator is specified, value will be returned inside a list. The optional parameter limit can be used to limit the number of split values in the result. If no limit is given, the number of splits returned is not bounded.

- REVERSE(value): Returns the reverse of the string value

- CONTAINS(text, search, return-index): Checks whether the string search is contained in the string text. By default, this function returns true if search is contained in text, and false otherwise. By passing true as the third function parameter return-index, the function will return the position of the first occurrence of search within text, starting at offset 0, or -1 if search is not contained in text.

The string matching performed by CONTAINS is case-sensitive.

- FIND_FIRST(text, search, start, end): Returns the position of the first occurrence of the string search inside the string text. Positions start at zero. If search is not contained in text, -1 is returned. The search can optionally be limited to a subset of text using the start and end arguments.

- FIND_LAST(text, search, start, end): Returns the position of the last occurrence of the string search inside the string text. Positions start at zero. If search is not contained in text, -1 is returned. The search can optionally be limited to a subset of text using the start and end arguments.

- LIKE(text, search, case-insensitive): Checks whether the pattern search is contained in the string text, using wildcard matching. Returns true if the pattern is contained in text, and false otherwise. The pattern string can contain the wildcard characters % (meaning any sequence of characters) and _ (any single character).

The string matching performed by LIKE is case-sensitive by default, but by passing true as the third parameter, the matching will be case-insensitive.

The value for search cannot be a variable or a document attribute. The actual value must be present at query parse time already.

- MD5(text): calculates the MD5 checksum for text and returns it in a hexadecimal string representation.

- SHA1(text): calculates the SHA1 checksum for text and returns it in a hexadecimal string representation.

- RANDOM_TOKEN(length): generates a pseudo-random token string with the specified length. length must be greater than zero and at most 65536. The generated token may consist of lower and uppercase letters and numbers. The algorithm for token generation should be treated as opaque.

Numeric functions

AQL offers some numeric functions for calculations. The following functions are supported:

- FLOOR(value): Returns the integer closest but not greater to value

- CEIL(value): Returns the integer closest but not less than value

- ROUND(value): Returns the integer closest to value

- ABS(value): Returns the absolute part of value

- SQRT(value): Returns the square root of value

- POW(base, exp): Returns the base to the exponent exp

- RAND(): Returns a pseudo-random number between 0 and 1

Date functions

AQL offers functionality to work with dates. Dates are no data types of their own in AQL (neither are they in JSON, which is often used as a format to ship data into and out of ArangoDB). Instead, dates in AQL are internally represented by either numbers (timestamps) or strings. The date functions in AQL provide mechanisms to convert from a numeric timestamp to a string representation and vice versa.

There are two date functions in AQL to create dates for further use:

- DATE_TIMESTAMP(date): Creates a UTC timestamp value from date. The return value has millisecond precision. To convert the return value to seconds, divide it by 1000.

- DATE_TIMESTAMP(year, month, day, hour, minute, second, millisecond): Same as before, but allows specifying the individual date components separately. All parameters after day are optional.

- DATE_ISO8601(date): Returns an ISO8601 date time string from date. The date time string will always use UTC time, indicated by the Z at its end.

- DATE_ISO8601(year, month, day, hour, minute, second, millisecond): same as before, but allows specifying the individual date components separately. All parameters after day are optional.

These two above date functions accept the following input values:

- numeric timestamps, indicating the number of milliseconds elapsed since the UNIX epoch (i.e. January 1st 1970 00:00:00 UTC). An example timestamp value is 1399472349522, which translates to 2014-05-07T14:19:09.522Z.

- date time strings in formats YYYY-MM-DDTHH:MM:SS.MMM, YYYY-MM-DD HH:MM:SS.MMM, or YYYY-MM-DD Milliseconds are always optional. A timezone difference may optionally be added at the end of the string, with the hours and minutes that need to be added or subtracted to the date time value. For example, 2014-05-07T14:19:09+01:00 can be used to specify a one hour offset, and 2014-05-07T14:19:09+07:30 can be specified for seven and half hours offset. Negative offsets are also possible. Alternatively to an offset, a Z can be used to indicate UTC / Zulu time.

An example value is 2014-05-07T14:19:09.522Z meaning May 7th 2014, 14:19:09 and 522 milliseconds, UTC / Zulu time. Another example value without time component is 2014-05-07Z.

Please note that if no timezone offset is specified in a date string, ArangoDB will assume UTC time automatically. This is done to ensure portability of queries across servers with different timezone settings, and because timestamps will always be UTC-based.

- individual date components as separate function arguments, in the following order:

- year

- month

- day

- hour

- minute

- second

- millisecond

All components following day are optional and can be omitted. Note that no timezone offsets can be specified when using separate date components, and UTC / Zulu time will be used.

The following calls to DATE_TIMESTAMP are equivalent and will all return 1399472349522:

DATE_TIMESTAMP("2014-05-07T14:19:09.522")

DATE_TIMESTAMP("2014-05-07T14:19:09.522Z")

DATE_TIMESTAMP("2014-05-07 14:19:09.522")

DATE_TIMESTAMP("2014-05-07 14:19:09.522Z")

DATE_TIMESTAMP(2014, 5, 7, 14, 19, 9, 522)

DATE_TIMESTAMP(1399472349522)

The same is true for calls to DATE_ISO8601 that also accepts variable input formats:

DATE_ISO8601("2014-05-07T14:19:09.522Z")

DATE_ISO8601("2014-05-07 14:19:09.522Z")

DATE_ISO8601(2014, 5, 7, 14, 19, 9, 522)

DATE_ISO8601(1399472349522)

The above functions are all equivalent and will return "2014-05-07T14:19:09.522Z".

The following date functions can be used with dates created by DATE_TIMESTAMP and DATE_ISO8601:

- DATE_DAYOFWEEK(date): Returns the weekday number of date. The return values have the following meanings:

- 0: Sunday

- 1: Monday

- 2: Tuesday

- 3: Wednesday

- 4: Thursday

- 5: Friday

- 6: Saturday

- DATE_YEAR(date): Returns the year part of date as a number.

- DATE_MONTH(date): Returns the month part of date as a number.

- DATE_DAY(date): Returns the day part of date as a number.

- DATE_HOUR(date): Returns the hour part of date as a number.

- DATE_MINUTE(date): Returns the minute part of date as a number.

- DATE_SECOND(date): Returns the seconds part of date as a number.

- DATE_MILLISECOND(date): Returns the milliseconds part of date as a number.

- DATE_DAYOFYEAR(date): Returns the day of year number of date. The return values range from 1 to 365, or 366 in a leap year respectively.

- DATE_ISOWEEK(date): Returns the ISO week date of date. The return values range from 1 to 53. Monday is considered the first day of the week. There are no fractional weeks, thus the last days in December may belong to the first week of the next year, and the first days in January may be part of the previous year's last week.

- DATE_LEAPYEAR(date): Returns whether the year of date is a leap year.

- DATE_QUARTER(date): Returns the quarter of the given date (1-based):

- 1: January, February, March

- 2: April, May, June

- 3: July, August, September

- 4: October, November, December

- DATE_DAYS_IN_MONTH(date): Returns the number of days in date's month (28..31).

The following other date functions are also available:

- DATE_NOW(): Returns the current time as a timestamp. The return value has millisecond precision. To convert the return value to seconds, divide it by 1000.

Note that this function is evaluated on every invocation and may return different values when invoked multiple times in the same query.

- DATE_ADD(date, amount, unit): Adds amount given in unit to date and returns the calculated date.

unit can be either of the following to specify the time unit to add or subtract (case-insensitive):

- y, year, years

- m, month, months

- w, week, weeks

- d, day, days

- h, hour, hours

- i, minute, minutes

- s, second, seconds

- f, millisecond, milliseconds

amount is the number of units to add (positive value) or subtract (negative value). It is recommended to use positive values only, and use  DATE_SUBTRACT()  for subtractions instead.

Examples:

DATE_ADD(DATE_NOW(), -1, "day") // yesterday; also see DATE_SUBTRACT()

DATE_ADD(DATE_NOW(), 3, "months") // in three months

DATE_ADD(DATE_ADD("2015-04-01", 5, "years"), 1, "month") // May 1st 2020

DATE_ADD("2015-04-01", 12*5 + 1, "months") // also May 1st 2020

DATE_ADD(DATE_TIMESTAMP(DATE_YEAR(DATE_NOW()), 12, 24), -4, "years") // Christmas four years ago

DATE_ADD(DATE_ADD("2016-02", "month", 1), -1, "day") // last day of February (29th, because 2016 is a leap year!)

You may also pass an ISO duration string as amount and leave out unit. The format is  P_Y_M_W_DT_H_M_._S , where underscores stand for digits and letters for time intervals - except for  P  (period) and  T  (time). The meaning of the other letters are:

- Y: years

- M: months (if before T)

- W: weeks

- D: days

- H: hours

- M: minutes (if after T)

- S: seconds (optionally with 3 decimal places for milliseconds)

The string must be prefixed by a  P . A separating  T  is only required if  H ,  M  and/or  S  are specified. You only need to specify the needed pairs of amounts and time intervals.

Examples:

DATE_ADD(DATE_NOW(), "P1Y") // add 1 year

DATE_ADD(DATE_NOW(), "P3M2W") // add 3 months and 2 weeks

DATE_ADD(DATE_NOW(), "P5DT26H") // add 5 days and 26 hours (=6 days and 2 hours)

DATE_ADD("2000-01-01", "PT4H") // add 4 hours

DATE_ADD("2000-01-01", "PT30M44.4S" // add 30 minutes, 44 seconds and 400 ms

DATE_ADD("2000-01-01", "P1Y2M3W4DT5H6M7.89S" // add a bit of everything

- DATE_SUBTRACT(date, amount, unit): Subtracts amount given in unit from date and returns the calculated date.

It works the same as DATE_ADD(), except that it subtracts. It is equivalent to calling DATE_ADD() with a negative amount, except that DATE_SUBTRACT() can also subtract ISO durations. Note that negative ISO durations are not supported (i.e. starting with  -P , like  -P1Y ).

Examples:

DATE_SUBTRACT(DATE_NOW(), 1, "day") // yesterday

DATE_SUBTRACT(DATE_TIMESTAMP(DATE_YEAR(DATE_NOW()), 12, 24), 4, "years") // Christmas four years ago

DATE_SUBTRACT(DATE_ADD("2016-02", "month", 1), 1, "day") // last day of February (29th, because 2016 is a leap year!)

DATE_SUBTRACT(DATE_NOW(), "P4D") // four days ago

DATE_SUBTRACT(DATE_NOW(), "PT1H3M") // 1 hour and 30 minutes ago

- DATE_DIFF(date1, date2, unit, asFloat): Calculate the difference between two dates in given time unit, optionally with decimal places. Returns a negative value if date1 is greater than / after date2.

unit can be either of the following to specify the time unit to return the difference in (case-insensitive):

- y, year, years

- m, month, months

- w, week, weeks

- d, day, days

- h, hour, hours

- i, minute, minutes

- s, second, seconds

- f, millisecond, milliseconds

- DATE_COMPARE(date1, date2, unitRangeStart, unitRangeEnd): Compare two partial dates and return true if they match, false otherwise. The parts to compare are defined by a range of time units.

The full range is: years, months, days, hours, minutes, seconds, milliseconds. Pass the unit to start from as unitRangeStart, and the unit to end with as unitRangeEnd. All units in between will be compared. Leave out unitRangeEnd to only compare unitRangeStart. You can refer to the units as:

- y, year, years

- m, month, months

- d, day, days

- h, hour, hours

- i, minute, minutes

- s, second, seconds

- f, millisecond, milliseconds

An error is raised if unitRangeEnd is a unit before unitRangeStart.

Examples:

// Compare months and days, true on birthdays if you're born on 4th of April

DATE_COMPARE("1985-04-04", DATE_NOW(), "months", "days")

// Will only match on one day if the current year is a leap year!

// You may want to add or subtract one day from date1 to match every year.

DATE_COMPARE("1984-02-29", DATE_NOW(), "months", days")

// compare years, months and days (true, because it's the same day)

DATE_COMPARE("2001-01-01T15:30:45.678Z", "2001-01-01T08:08:08.008Z", "years", "days")

You can directly compare ISO date strings if you want to find dates before or after a certain date, or in between two dates ( >= ,  > ,  < ,  <= ). No special date function is required. Equality tests ( ==  and  != ) will only match the exact same date and time however. You may use  SUBSTRING()  to compare partial date strings,  DATE_COMPARE()  is basically a convenience function for that. However, neither is really required to limit a search to a certain day for instance:

FOR doc IN coll

FILTER doc.date >= "2015-05-15" AND doc.date < "2015-05-16"

RETURN doc

Every ISO date on that day is greater than  2015-05-15  in a string comparison (e.g.  2015-05-15T11:30:00.000Z ). The time components will be "ignored". The equal sign in  >=  merely helps to express the semantic. Dates before  2015-05-15  are less and therefore filtered out. The second condition works likewise. The query will return every document with  date  ranging from  2015-05-15T00:00:00.000Z  to  2015-05-15T23:99:99.999Z . It would also include  2015-05-15T24:00:00.000Z , but that date is actually  2015-05-16T00:00:00.000Z  and can only occur if inserted manually.

Leap days in leap years (29th of February) must be always handled manually, if you require so (e.g. birthday checks):

LET today = DATE_NOW()

LET noLeapYear = NOT DATE_LEAPYEAR(today)

FOR user IN users

LET birthday = noLeapYear AND

DATE_MONTH(user.birthday) == 2 AND

DATE_DAY(user.birthday) == 29

? DATE_SUBTRACT(user.birthday, 1, "day") /* treat like 28th in non-leap years */

: user.birthday

FILTER DATE_COMPARE(today, birthday, "month", "day")

/* includes leaplings on the 28th of February in non-leap years,

* but excludes them in leap years which do have a 29th February.

* Replace DATE_SUBTRACT() by DATE_ADD() to include them on the 1st of March

* in non-leap years instead (depends on local jurisdiction).

*/

RETURN user

- DATE_FORMAT(date, format): Format a date according to the given format string. It supports the following placeholders (case-insensitive):

- %t: timestamp, in milliseconds since midnight 1970-01-01

- %z: ISO date (0000-00-00T00:00:00.000Z)

- %w: day of week (0..6)

- %y: year (0..9999)

- %yy: year (00..99), abbreviated (last two digits)

- %yyyy: year (0000..9999), padded to length of 4

- %yyyyyy: year (-009999 .. +009999), with sign prefix and padded to length of 6

- %m: month (1..12)

- %mm: month (01..12), padded to length of 2

- %d: day (1..31)

- %dd: day (01..31), padded to length of 2

- %h: hour (0..23)

- %hh: hour (00..23), padded to length of 2

- %i: minute (0..59)

- %ii: minute (00..59), padded to length of 2

- %s: second (0..59)

- %ss: second (00..59), padded to length of 2

- %f: millisecond (0..999)

- %fff: millisecond (000..999), padded to length of 3

- %x: day of year (1..366)

- %xxx: day of year (001..366), padded to length of 3

- %k: ISO week date (1..53)

- %kk: ISO week date (01..53), padded to length of 2

- %l: leap year (0 or 1)

- %q: quarter (1..4)

- %a: days in month (28..31)

- %mmm: abbreviated English name of month (Jan..Dec)

- %mmmm: English name of month (January..December)

- %www: abbreviated English name of weekday (Sun..Sat)

- %wwww: English name of weekday (Sunday..Saturday)

- %&: special escape sequence for rare occasions

- %%: literal %

- %: ignored

%yyyy  does not enforce a length of 4 for years before 0 and past 9999. The same format as for  %yyyyyy  will used instead.  %yy  preserves the sign for negative years and thus returns 3 characters in total.

Single  %  characters will be ignored. Use  %%  for a literal  % . To resolve ambiguities like in  %mmonth  (unpadded month number + the string "month") between  %mm  + "onth" and  %m  + "month", use the escape sequence  %& :  %m%&month .

Note that this is a rather costly operation and may not be suitable for large datasets (like over 1 million dates). If possible, avoid formatting dates on server-side and leave it up to the client to do so. This function should only be used for special date comparisons or to store the formatted dates in the database. For better performance, use the primitive  DATE_*()  functions together with  CONCAT()  if possible.

Examples:

DATE_FORMAT(DATE_NOW(), "%q/%yyyy") // quarter and year (e.g. "3/2015")

DATE_FORMAT(DATE_NOW(), "%dd.%mm.%yyyy %hh:%ii:%ss.%fff") // e.g. "18.09.2015 15:30:49.374"

DATE_FORMAT("1969", "Summer of '%yy") // "Summer of '69"

DATE_FORMAT("2016", "%%l = %l") // "%l = 1" (2016 is a leap year)

DATE_FORMAT("2016-03-01", "%xxx%") // "063", trailing % ignored

- IS_DATESTRING(value): Returns true if value is a string that can be used in a date function. This includes partial dates such as 2015 or 2015-10 and strings containing invalid dates such as 2015-02-31. The function will return false for all non-string values, even if some of them may be usable in date functions.

Working with dates and indices

There are two recommended ways to store timestamps in ArangoDB:

- as string with ISO 8601 UTC timestamp

- as Epoch number

The sort order of both is identical due to the sort properties of ISO date strings. Therefore, you can work with skiplist indices and use string comparisons (less than, greater than, in, equality) to express time ranges in your queries:

arangosh> db._create("exampleTime");

arangosh> var timestamps = ["2014-05-07T14:19:09.522","2014-05-07T21:19:09.522","2014-05-08T04:19:09.522","2014-05-08T11:19:09.522","2014-05-08T18:19:09.522"];

arangosh> for (i = 0; i < 5; i++) db.exampleTime.save({value:i, ts: timestamps[i]})

arangosh> db._query("FOR d IN exampleTime FILTER d.ts > '2014-05-07T14:19:09.522' and d.ts < '2014-05-08T18:19:09.522' RETURN d").toArray()

show execution results

The first and the last timestamp in the array are excluded from the result by the  FILTER .

Limitations

Note that dates before the year 1583 aren't allowed by the ISO 8601 standard by default, because they lie before the official introduction of the Gregorian calendar and may thus be incorrect or invalid. All AQL date functions apply the same rules to every date according to the Gregorian calendar system, even if inappropriate. That does not constitute a problem, unless you deal with dates prior to 1583 and especially years before Christ. The standard allows negative years, but requires special treatment of positive years too, if negative years are used (e.g.  +002015-05-15  and  -000753-01-01 ). This is rarely used however, and AQL does not use the 7-character version for years between 0 and 9999 in ISO strings. Keep in mind that they can't be properly compared to dates outside that range. Leap seconds are ignored, just as they are in JavaScript as per ECMAScript Language Specifications.

Array functions

AQL supports the following functions to operate on array values:

- LENGTH(array): Returns the length (number of array elements) of array. If array is an object / document, returns the number of attribute keys of the document, regardless of their values. If array is a collection, returns the number of documents in it.

- FLATTEN(array, depth): Turns an array of arrays into a flat array. All array elements in array will be expanded in the result array. Non-array elements are added as they are. The function will recurse into sub-arrays up to a depth of depth. depth has a default value of 1.

Examples

FLATTEN([ 1, 2, [ 3, 4 ], 5, [ 6, 7 ], [ 8, [ 9, 10 ] ] ])

will produce:

[ 1, 2, 3, 4, 5, 6, 7, 8, [ 9, 10 ] ]

To fully flatten the array, use a depth of 2:

FLATTEN([ 1, 2, [ 3, 4 ], 5, [ 6, 7 ], [ 8, [ 9, 10 ] ] ], 2)

This will produce:

[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ]

- MIN(array): Returns the smallest element of array. null values are ignored. If the array is empty or only null values are contained in the array, the function will return null.

- MAX(array): Returns the greatest element of array. null values are ignored. If the array is empty or only null values are contained in the array, the function will return null.

- AVERAGE(array): Returns the average (arithmetic mean) of the values in array. This requires the elements in array to be numbers. null values are ignored. If the array is empty or only null values are contained in the array, the function will return null.

- SUM(array): Returns the sum of the values in array. This requires the elements in array to be numbers. null values are ignored.

- MEDIAN(array): Returns the median value of the values in array. This requires the elements in array to be numbers. null values are ignored. If the array is empty or only null values are contained in the array, the function will return null.

- PERCENTILE(array, n, method): Returns the nth percentile of the values in array. This requires the elements in array to be numbers. null values are ignored. n must be between 0 (excluded) and 100 (included). method can be rank or interpolation. The function will return null if the array is empty or only null values are contained in it or the percentile cannot be calculated.

- VARIANCE_POPULATION(array): Returns the population variance of the values in array. This requires the elements in array to be numbers. null values are ignored. If the array is empty or only null values are contained in the array, the function will return null.

- VARIANCE_SAMPLE(array): Returns the sample variance of the values in array. This requires the elements in array to be numbers. null values are ignored. If the array is empty or only null values are contained in the array, the function will return null.

- STDDEV_POPULATION(array): Returns the population standard deviation of the values in array. This requires the elements in array to be numbers. null values are ignored. If the array is empty or only null values are contained in the array, the function will return null.

- STDDEV_SAMPLE(array): Returns the sample standard deviation of the values in array. This requires the elements in array to be numbers. null values are ignored. If the array is empty or only null values are contained in the array, the function will return null.

- REVERSE(array): Returns the elements in array in reversed order.

- FIRST(array): Returns the first element in array or null if the array is empty.

- LAST(array): Returns the last element in array or null if the array is empty.

- NTH(array, position): Returns the array element at position position. Positions start at 0. If position is negative or beyond the upper bound of the array specified by array, then null will be returned.

- POSITION(array, search, return-index): Returns the position of the element search in array. Positions start at 0. If the element is not found, then -1 is returned. If return-index is false, then instead of the position only true or false are returned, depending on whether the sought element is contained in the array.

- SLICE(array, start, length): Extracts a slice of the array specified by array. The extraction will start at array element with position start. Positions start at 0. Up to length elements will be extracted. If length is not specified, all array elements starting at start will be returned. If start is negative, it can be used to indicate positions from the end of the array.

Examples

SLICE([ 1, 2, 3, 4, 5 ], 0, 1)

will return [ 1 ]

SLICE([ 1, 2, 3, 4, 5 ], 1, 2)

will return [ 2, 3 ]

SLICE([ 1, 2, 3, 4, 5 ], 3)

will return [ 4, 5 ]

SLICE([ 1, 2, 3, 4, 5 ], 1, -1)

will return [ 2, 3, 4 ]

SLICE([ 1, 2, 3, 4, 5 ], 0, -2)

will return [ 1, 2, 3 ]

- UNIQUE(array): Returns all unique elements in array. To determine uniqueness, the function will use the comparison order. Calling this function may return the unique elements in any order.

- UNION(array1, array2, ...): Returns the union of all arrays specified. The function expects at least two array values as its arguments. The result is an array of values in an undefined order.

Note: No duplicates will be removed. In order to remove duplicates, please use either UNION_DISTINCT function or apply the UNIQUE on the result of union.

Examples

RETURN UNION(

[ 1, 2, 3 ],

[ 1, 2 ]

)

will produce:

[ [ 1, 2, 3, 1, 2 ] ]

with duplicate removal:

RETURN UNIQUE(

UNION(

[ 1, 2, 3 ],

[ 1, 2 ]

)

)

will produce:

[ [ 1, 2, 3 ] ]

- UNION_DISTINCT(array1, array2, ...): Returns the union of distinct values of all arrays specified. The function expects at least two array values as its arguments. The result is an array of values in an undefined order.

- MINUS(array1, array2, ...): Returns the difference of all arrays specified. The function expects at least two array values as its arguments. The result is an array of values that occur in the first array but not in any of the subsequent arrays. The order of the result array is undefined and should not be relied on. Note: duplicates will be removed.

- INTERSECTION(array1, array2, ...): Returns the intersection of all arrays specified. The function expects at least two array values as its arguments. The result is an array of values that occur in all arguments. The order of the result array is undefined and should not be relied on.

Note: Duplicates will be removed.

- APPEND(array, values, unique): Adds all elements from the array values to the array specified by array. If unique is set to true, then only those values will be added that are not already contained in array. The modified array is returned. All values are added at the end of the array (right side).

/* [ 1, 2, 3, 5, 6, 9 ] */

APPEND([ 1, 2, 3 ], [ 5, 6, 9 ])

/* [ 1, 2, 3, 4, 5, 9 ] */

APPEND([ 1, 2, 3 ], [ 3, 4, 5, 2, 9 ], true)

- PUSH(array, value, unique): Adds value to the array specified by array. If unique is set to true, then value is not added if already present in the array. The modified array is returned. The value is added at the end of the array (right side).

Note: non-unique elements will not be removed from the array if they were already present before the call to  PUSH . The unique flag will only control if the value will be added again to the array if already present. To make a array unique, use the  UNIQUE  function.

/* [ 1, 2, 3, 4 ] */

PUSH([ 1, 2, 3 ], 4)

/* [ 1, 2, 3 ] */

PUSH([ 1, 2, 3 ], 2, true)

- UNSHIFT(array, value, unique): Adds value to the array specified by array. If unique is set to true, then value is not added if already present in the array. The modified array is returned. The value is added at the start of the array (left side).

Note: non-unique elements will not be removed from the array if they were already present before the call to  UNSHIFT . The unique flag will only control if the value will be added again to the array if already present. To make a array unique, use the  UNIQUE  function.

/* [ 4, 1, 2, 3 ] */

UNSHIFT([ 1, 2, 3 ], 4)

/* [ 1, 2, 3 ] */

UNSHIFT([ 1, 2, 3 ], 2, true)

- POP(array): Removes the element at the end (right side) of array. The modified array is returned. If the array is already empty or null, an empty array is returned.

/* [ 1, 2, 3 ] */

POP([ 1, 2, 3, 4 ])

- SHIFT(array): Removes the element at the start (left side) of array. The modified array is returned. If the array is already empty or null, an empty array is returned.

/* [ 2, 3, 4 ] */

SHIFT([ 1, 2, 3, 4 ])

- REMOVE_VALUE(array, value, limit): Removes all occurrences of value in the array specified by array. If the optional limit is specified, only limit occurrences will be removed.

/* [ "b", "b", "c" ] */

REMOVE_VALUE([ "a", "b", "b", "a", "c" ], "a")

/* [ "b", "b", "a", "c" ] */

REMOVE_VALUE([ "a", "b", "b", "a", "c" ], "a", 1)

- REMOVE_VALUES(array, values): Removes all occurrences of any of the values specified in array values from the array specified by array.

/* [ "b", "c", "e", "g" ] */

REMOVE_VALUES([ "a", "b", "c", "d", "e", "f", "g" ], [ "a", "f", "d" ])

- REMOVE_NTH(array, position): Removes the element at position position from the array specified by array. Positions start at 0. Negative positions are supported, with -1 being the last array element. If position is out of bounds, the array is returned unmodified. Otherwise, the modified array is returned.

/* [ "a", "c", "d", "e" ] */

REMOVE_NTH([ "a", "b", "c", "d", "e" ], 1)

/* [ "a", "b", "c", "e" ] */

REMOVE_NTH([ "a", "b", "c", "d", "e" ], -2)

Apart from these functions, AQL also offers several language constructs (e.g. FOR, SORT, LIMIT, COLLECT) to operate on arrays.

Document functions

AQL supports the following functions to operate on document values:

- MATCHES(document, examples, return-index): Compares the document document against each example document provided in the array examples. If document matches one of the examples, true is returned, and if there is no match false will be returned. The default return value type can be changed by passing true as the third function parameter return-index. Setting this flag will return the index of the example that matched (starting at offset 0), or -1 if there was no match.

The comparisons will be started with the first example. All attributes of the example will be compared against the attributes of document. If all attributes match, the comparison stops and the result is returned. If there is a mismatch, the function will continue the comparison with the next example until there are no more examples left.

The examples must be an array of 1..n example documents, with any number of attributes each. Note: specifying an empty array of examples is not allowed.

Examples

MATCHES(

{ "test" : 1 }, [

{ "test" : 1, "foo" : "bar" },

{ "foo" : 1 },

{ "test : 1 }

], true)

This will return 2, because the third example matches, and because the return-index flag is set to true.

- MERGE(document1, document2, ... documentN): Merges the documents in document1 to documentN into a single document. If document attribute keys are ambiguous, the merged result will contain the values of the documents contained later in the argument list.

For example, two documents with distinct attribute names can easily be merged into one:

/* { "user1" : { "name" : "J" }, "user2" : { "name" : "T" } } */

MERGE(

{ "user1" : { "name" : "J" } },

{ "user2" : { "name" : "T" } }

)

When merging documents with identical attribute names, the attribute values of the latter documents will be used in the end result:

/* { "users" : { "name" : "T" } } */

MERGE(

{ "users" : { "name" : "J" } },

{ "users" : { "name" : "T" } }

)

MERGE works with a single array parameter, too. This variant allows combining the attributes of multiple objects from the array into a single object, e.g.

RETURN MERGE([

{ foo: 'bar' },

{ quux: 'quetzalcoatl', ruled: true },

{ bar: 'baz', foo: 'done' }

])

This will now return:

{

"foo": "done",

"quux": "quetzalcoatl",

"ruled": true,

"bar": "baz"

}

Please note that merging will only be done for top-level attributes. If you wish to merge sub-attributes, you should consider using MERGE_RECURSIVE instead.

- MERGE_RECURSIVE(document1, document2, ... documentN): Recursively merges the documents in document1 to documentN into a single document. If document attribute keys are ambiguous, the merged result will contain the values of the documents contained later in the argument list.

For example, two documents with distinct attribute names can easily be merged into one:

/* { "user-1" : { "name" : "J", "livesIn" : { "city" : "LA", "state" : "CA" }, "age" : 42 } } */

MERGE_RECURSIVE(

{ "user-1" : { "name" : "J", "livesIn" : { "city" : "LA" } } },

{ "user-1" : { "age" : 42, "livesIn" : { "state" : "CA" } } }

)

MERGE_RECURSIVE does not support the single array parameter variant that MERGE offers.

- TRANSLATE(value, lookup, defaultValue): Looks up the value value in the lookup document. If value is a key in lookup, then value will be replaced with the lookup value found. If value is not present in lookup, then defaultValue will be returned if specified. If no defaultValue is specified, value will be returned:

/* "France" */

TRANSLATE("FR", { US: "United States", UK: "United Kingdom", FR: "France" })

/* "not found!" */

TRANSLATE(42, { foo: "bar", bar: "baz" }, "not found!")

- HAS(document, attributename): Returns true if document has an attribute named attributename, and false otherwise.

- ATTRIBUTES(document, removeInternal, sort): Returns the attribute names of the document as an array. If removeInternal is set to true, then all internal attributes (such as _id, _key etc.) are removed from the result. If sort is set to true, then the attribute names in the result will be sorted. Otherwise they will be returned in any order.

- VALUES(document, removeInternal): Returns the attribute values of the document as an array. If removeInternal is set to true, then all internal attributes (such as _id, _key etc.) are removed from the result.

- ZIP(attributes, values): Returns a document object assembled from the separate parameters attributes and values. attributes and values must be arrays and must have the same length. The items in attributes will be used for naming the attributes in the result. The items in values will be used as the actual values of the result.

/* { "name" : "some user", "active" : true, "hobbies" : [ "swimming", "riding" ] } */

ZIP([ 'name', 'active', 'hobbies' ], [ 'some user', true, [ 'swimming', 'riding' ] ])

- UNSET(document, attributename, ...): Removes the attributes attributename (can be one or many) from document. All other attributes will be preserved. Multiple attribute names can be specified by either passing multiple individual string argument names, or by passing an array of attribute names:

UNSET(doc, '_id', '_key', 'foo', 'bar')

UNSET(doc, [ '_id', '_key', 'foo', 'bar' ])

- UNSET_RECURSIVE(document, attributename, ...): Recursively removes the attributes attributename (can be one or many) from document and its sub-documents. All other attributes will be preserved. Multiple attribute names can be specified by either passing multiple individual string argument names, or by passing an array of attribute names:

UNSET_RECURSIVE(doc, '_id', '_key', 'foo', 'bar')

UNSET_RECURSIVE(doc, [ '_id', '_key', 'foo', 'bar' ])

- KEEP(document, attributename, ...): Keeps only the attributes attributename (can be one or many) from document. All other attributes will be removed from the result. Multiple attribute names can be specified by either passing multiple individual string argument names, or by passing an array of attribute names:

KEEP(doc, 'firstname', 'name', 'likes')

KEEP(doc, [ 'firstname', 'name', 'likes' ])

- PARSE_IDENTIFIER(document-handle): Parses the document handle specified in document-handle and returns a the handle's individual parts a separate attributes. This function can be used to easily determine the collection name and key from a given document. The document-handle can either be a regular document from a collection, or a document identifier string (e.g. _users/1234). Passing either a non-string or a non-document or a document without an _id attribute will result in an error.

/* { "collection" : "_users", "key" : "my-user" } */

PARSE_IDENTIFIER('_users/my-user')

/* { "collection" : "mycollection", "key" : "mykey" } */

PARSE_IDENTIFIER({ "_id" : "mycollection/mykey", "value" : "some value" })

- IS_SAME_COLLECTION(collection, document): Return true if document has the same collection id as the collection specified in collection. document can either be a document handle string, or a document with an _id attribute. The function does not validate whether the collection actually contains the specified document, but only compares the name of the specified collection with the collection name part of the specified document. If document is neither an object with an id attribute nor a string value, the function will return null and raise a warning.

/* true */

IS_SAME_COLLECTION('_users', '_users/my-user')

IS_SAME_COLLECTION('_users', { _id: '_users/my-user' })

/* false */

IS_SAME_COLLECTION('_users', 'foobar/baz')

IS_SAME_COLLECTION('_users', { _id: 'something/else' })

Geo functions

AQL offers the following functions to filter data based on geo indexes:

- NEAR(collection, latitude, longitude, limit, distancename): Returns at most limit documents from collection collection that are near latitude and longitude. The result contains at most limit documents, returned in any order. If more than limit documents qualify, it is undefined which of the qualifying documents are returned. Optionally, the distances between the specified coordinate (latitude and longitude) and the document coordinates can be returned as well. To make use of that, an attribute name for the distance result has to be specified in the distancename argument. The result documents will contain the distance value in an attribute of that name. limit is an optional parameter since ArangoDB 1.3. If it is not specified or null, a limit value of 100 will be applied.

- WITHIN(collection, latitude, longitude, radius, distancename): Returns all documents from collection collection that are within a radius of radius around that specified coordinate (latitude and longitude). The order in which the result documents are returned is undefined. Optionally, the distance between the coordinate and the document coordinates can be returned as well. To make use of that, an attribute name for the distance result has to be specified in the distancename argument. The result documents will contain the distance value in an attribute of that name.

- WITHIN_RECTANGLE(collection, latitude1, longitude1, latitude2, longitude2): Returns all documents from collection collection that are positioned inside the bounding rectangle with the points (latitude1, longitude1) and (latitude2, longitude2).

Note: these functions require the collection collection to have at least one geo index. If no geo index can be found, calling this function will fail with an error.

- IS_IN_POLYGON(polygon, latitude, longitude): Returns  true  if the point (latitude, longitude) is inside the polygon specified in the polygon parameter. The result is undefined (may be  true  or  false ) if the specified point is exactly on a boundary of the polygon.

latitude can alternatively be specified as an array with two values. By default, the first array element will be interpreted as the latitude value and the second array element as the longitude value. This can be changed by setting the 3rd parameter to  true . polygon needs to be an array of points, with each point being an array with two values. The first value of each point is considered to be the latitude value and the second to be the longitude value, unless the 3rd parameter has a value of  true . That means latitude and longitude need to be specified in the same order in the points parameter as they are in the search coordinate.

Examples:

/* will check if the point (lat 4, lon 7) is contained inside the polygon */

RETURN IS_IN_POLYGON([ [ 0, 0 ], [ 0, 10 ], [ 10, 10 ], [ 10, 0 ] ], 4, 7)

/* will check if the point (lat 4, lon 7) is contained inside the polygon */

RETURN IS_IN_POLYGON([ [ 0, 0 ], [ 0, 10 ], [ 10, 10 ], [ 10, 0 ] ], [ 4, 7 ])

/* will check if the point (lat 4, lon 7) is contained inside the polygon */

RETURN IS_IN_POLYGON([ [ 0, 0 ], [ 10, 0 ], [ 10, 10 ], [ 0, 10 ] ], [ 7, 4 ], true)

Fulltext functions

AQL offers the following functions to filter data based on fulltext indexes:

- FULLTEXT(collection, attribute, query, limit): Returns all documents from collection collection for which the attribute attribute matches the fulltext query query. The limit parameter is optional. If set to a non-zero value, it will cap the result to at most this number of documents. query is a comma-separated list of sought words (or prefixes of sought words). To distinguish between prefix searches and complete-match searches, each word can optionally be prefixed with either the prefix: or complete: qualifier. Different qualifiers can be mixed in the same query. Not specifying a qualifier for a search word will implicitly execute a complete-match search for the given word:

- FULLTEXT(emails, "body", "banana") Will look for the word banana in the attribute body of the collection collection.

- FULLTEXT(emails, "body", "banana,orange") Will look for both words banana and orange in the mentioned attribute. Only those documents will be returned that contain both words.

- FULLTEXT(emails, "body", "prefix:head") Will look for documents that contain any words starting with the prefix head.

- FULLTEXT(emails, "body", "prefix:head,complete:aspirin") Will look for all documents that contain a word starting with the prefix head and that also contain the (complete) word aspirin. Note: specifying complete is optional here.

- FULLTEXT(emails, "body", "prefix:cent,prefix:subst") Will look for all documents that contain a word starting with the prefix cent and that also contain a word starting with the prefix subst.

If multiple search words (or prefixes) are given, then by default the results will be AND-combined, meaning only the logical intersection of all searches will be returned. It is also possible to combine partial results with a logical OR, and with a logical NOT:

- FULLTEXT(emails, "body", "+this,+text,+document") Will return all documents that contain all the mentioned words. Note: specifying the + symbols is optional here.

- FULLTEXT(emails, "body", "banana,|apple") Will return all documents that contain either (or both) words banana or apple.

- FULLTEXT(emails, "body", "banana,-apple") Will return all documents that contain the word banana but do not contain the word apple.

- FULLTEXT(emails, "body", "banana,pear,-cranberry") Will return all documents that contain both the words banana and pear but do not contain the word cranberry.

No precedence of logical operators will be honored in a fulltext query. The query will simply be evaluated from left to right.

Note: the FULLTEXT function requires the collection collection to have a fulltext index on attribute. If no fulltext index is available, this function will fail with an error. FULLTEXT is not meant to be used as an argument to FILTER but rather to be used as the expression of the FOR statement:

FOR oneMail IN FULLTEXT(emails, "body", "banana,-apple") RETURN oneMail._id;

Control flow functions

AQL offers the following functions to let the user control the flow of operations:

- NOT_NULL(alternative, ...): Returns the first alternative that is not null, and null if all alternatives are null themselves

- FIRST_LIST(alternative, ...): Returns the first alternative that is an array, and null if none of the alternatives is an array

- FIRST_DOCUMENT(alternative, ...): Returns the first alternative that is a document, and null if none of the alternatives is a document

Miscellaneous functions

Finally, AQL supports the following functions that do not belong to any of the other function categories:

- COLLECTIONS(): Returns an array of collections. Each collection is returned as a document with attributes name and _id

- CURRENT_USER(): Returns the name of the current user. The current user is the user account name that was specified in the Authorization HTTP header of the request. It will only be populated if authentication on the server is turned on, and if the query was executed inside a request context. Otherwise, the return value of this function will be null.

- DOCUMENT(collection, id): Returns the document which is uniquely identified by the id. ArangoDB will try to find the document using the _id value of the document in the specified collection. If there is a mismatch between the collection passed and the collection specified in id, then null will be returned. Additionally, if the collection matches the collection value specified in id but the document cannot be found, null will be returned. This function also allows id to be an array of ids. In this case, the function will return an array of all documents that could be found.

Examples

DOCUMENT(users, "users/john")

DOCUMENT(users, "john")

DOCUMENT(users, [ "users/john", "users/amy" ])

DOCUMENT(users, [ "john", "amy" ])

Note: The DOCUMENT function is polymorphic since ArangoDB 1.4. It can now be used with a single parameter id as follows:

- DOCUMENT(id): In this case, id must either be a document handle string (consisting of collection name and document key) or an array of document handle strings, e.g.

DOCUMENT("users/john")

DOCUMENT([ "users/john", "users/amy" ])

- CALL(function, arg1, ..., argN): Dynamically calls the function with name function with the arguments specified. Both built-in and user-defined functions can be called. Arguments are passed as separate parameters to the called function.

/* "this" */

CALL('SUBSTRING', 'this is a test', 0, 4)

- APPLY(function, arguments): Dynamically calls the function with name function with the arguments specified. Both built-in and user-defined functions can be called. Arguments are passed as separate parameters to the called function.

/* "this is" */

APPLY('SUBSTRING', [ 'this is a test', 0, 7 ])

Query results

Result sets

The result of an AQL query is an array of values. The individual values in the result array may or may not have a homogeneous structure, depending on what is actually queried.

For example, when returning data from a collection with inhomogeneous documents (the individual documents in the collection have different attribute names) without modification, the result values will as well have an inhomogeneous structure. Each result value itself is a document:

FOR u IN users

RETURN u

[ { "id" : 1, "name" : "John", "active" : false },

{ "age" : 32, "id" : 2, "name" : "Vanessa" },

{ "friends" : [ "John", "Vanessa" ], "id" : 3, "name" : "Amy" } ]

However, if a fixed set of attributes from the collection is queried, then the query result values will have a homogeneous structure. Each result value is still a document:

FOR u IN users

RETURN { "id" : u.id, "name" : u.name }

[ { "id" : 1, "name" : "John" },

{ "id" : 2, "name" : "Vanessa" },

{ "id" : 3, "name" : "Amy" } ]

It is also possible to query just scalar values. In this case, the result set is an array of scalars, and each result value is a scalar value:

FOR u IN users

RETURN u.id

[ 1, 2, 3 ]

If a query does not produce any results because no matching data can be found, it will produce an empty result array:

[ ]

Errors

Issuing an invalid query to the server will result in a parse error if the query is syntactically invalid. ArangoDB will detect such errors during query inspection and abort further processing. Instead, the error number and an error message are returned so that the errors can be fixed.

If a query passes the parsing stage, all collections referenced in the query will be opened. If any of the referenced collections is not present, query execution will again be aborted and an appropriate error message will be returned.

Under some circumstances, executing a query may also produce run-time errors that cannot be predicted from inspecting the query text alone. This is because queries may use data from collections that may also be inhomogeneous. Some examples that will cause run-time errors are:

- Division by zero: Will be triggered when an attempt is made to use the value 0 as the divisor in an arithmetic division or modulus operation

- Invalid operands for arithmetic operations: Will be triggered when an attempt is made to use any non-numeric values as operands in arithmetic operations. This includes unary (unary minus, unary plus) and binary operations (plus, minus, multiplication, division, and modulus)

- Invalid operands for logical operations: Will be triggered when an attempt is made to use any non-boolean values as operand(s) in logical operations. This includes unary (logical not/negation), binary (logical and, logical or), and the ternary operators

Please refer to the Arango Errors page for a list of error codes and meanings.

Operators

AQL supports a number of operators that can be used in expressions. There are comparison, logical, arithmetic, and the ternary operator.

Comparison operators

Comparison (or relational) operators compare two operands. They can be used with any input data types, and will return a boolean result value.

The following comparison operators are supported:

- == equality

- != inequality

- < less than

- <= less or equal

- > greater than

- >= greater or equal

- IN test if a value is contained in an array

- NOT IN test if a value is not contained in an array

These operators accept any data types for the first and second operands.

Each of the comparison operators returns a boolean value if the comparison can be evaluated and returns true if the comparison evaluates to true, and false otherwise. Please note that the comparison operators will not perform any implicit type casts if the compared operands have different types.

Some examples for comparison operations in AQL:

0 == null                 // false

1 > 0                     // true

true != null              // true

45 <= "yikes!"            // true

65 != "65"                // true

65 == 65                  // true

1.23 > 1.32               // false

1.5 IN [ 2, 3, 1.5 ]      // true

"foo" IN null             // false

42 NOT IN [ 17, 40, 50 ]  // true

Logical operators

The following logical operators are supported in AQL:

- && logical and operator

- || logical or operator

- ! logical not/negation operator

AQL also supports the following alternative forms for the logical operators:

- AND logical and operator

- OR logical or operator

- NOT logical not/negation operator

The alternative forms are aliases and functionally equivalent to the regular operators.

The two-operand logical operators in AQL will be executed with short-circuit evaluation. The result of the logical operators in AQL is defined as follows:

- lhs && rhs  will return  lhs  if it is  false  or would be  false  when converted into a boolean. If  lhs  is  true  or would be  true  when converted to a boolean,  rhs  will be returned.

- lhs || rhs  will return  lhs  if it is  true  or would be  true  when converted into a boolean. If  lhs  is  false  or would be  false  when converted to a boolean,  rhs  will be returned.

- ! value  will return the negated value of  value  converted into a boolean

Some examples for logical operations in AQL:

u.age > 15 && u.address.city != ""

true || false

! u.isInvalid

1 || ! 0

Older versions of ArangoDB required the operands of all logical operators to be boolean values and failed when non-boolean values were passed into the operators. Additionally, the result of any logical operation always was a boolean value.

This behavior has changed in ArangoDB 2.3. Passing non-boolean values to a logical operator is now allowed. Any-non boolean operands will be casted to boolean implicitly by the operator, without making the query abort.

The conversion to a boolean value works as follows:

- null  will be converted to  false

- boolean values remain unchanged

- all numbers unequal to zero are  true , zero is  false

- the empty string is  false , all other strings are  true

- arrays ( [ ] ) and objects / documents ( { } ) are  true , regardless of their contents

The result of logical and and logical or operations can now have any data type and is not necessarily a boolean value.

For example, the following logical operations will return a boolean values:

25 > 1 && 42 != 7                          // true

22 IN [ 23, 42 ] || 23 NOT IN [ 22, 7 ]    // true

25 != 25                                   // false

whereas the following logical operations will not return boolean values:

1 || 7                                     // 1

null || "foo"                              // "foo"

null && true                               // null

true && 23                                 // 23

Arithmetic operators

Arithmetic operators perform an arithmetic operation on two numeric operands. The result of an arithmetic operation is again a numeric value. Operators are supported.

AQL supports the following arithmetic operators:

- + addition

- - subtraction

- \* multiplication

- / division

- % modulus

The unary plus and unary minus are supported as well.

Some example arithmetic operations:

1 + 1

33 - 99

12.4 * 4.5

13.0 / 0.1

23 % 7

-15

+9.99

The arithmetic operators accept operands of any type. This behavior has changed in ArangoDB 2.3. Passing non-numeric values to an arithmetic operator is now allow. Any-non numeric operands will be casted to numbers implicitly by the operator, without making the query abort.

The conversion to a numeric value works as follows:

- null  will be converted to  0

- false  will be converted to  0 , true will be converted to  1

- a valid numeric value remains unchanged, but NaN and Infinity will be converted to  null

- string values are converted to a number if they contain a valid string representation of a number. Any whitespace at the start or the end of the string is ignored. Strings with any other contents are converted to  null

- an empty array is converted to  0 , an array with one member is converted to the numeric representation of its sole member. Arrays with more members are converted to  null

- objects / documents are converted to  null

If the conversion to a number produces a value of  null  for one of the operands, the result of the whole arithmetic operation will also be  null . An arithmetic operation that produces an invalid value, such as  1 / 0  will also produce a value of  null .

Here are a few examples:

1 + "a"                 // null

1 + "99"                // 100

1 + null                // 1

null + 1                // 1

3 + [ ]                 // 3

24 + [ 2 ]              // 26

24 + [ 2, 4 ]           // null

25 - null               // 25

17 - true               // 16

23 * { }                // null

5 * [ 7 ]               // 35

24 / "12"               // 2

1 / 0                   // null

Ternary operator

AQL also supports a ternary operator that can be used for conditional evaluation. The ternary operator expects a boolean condition as its first operand, and it returns the result of the second operand if the condition evaluates to true, and the third operand otherwise.

Examples

u.age > 15 || u.active == true ? u.userId : null

Range operator

AQL supports expressing simple numeric ranges with the .. operator. This operator can be used to easily iterate over a sequence of numeric values.

The .. operator will produce an array of values in the defined range, with both bounding values included.

Examples

2010..2013

will produce the following result:

[ 2010, 2011, 2012, 2013 ]

Operator precedence

The operator precedence in AQL is similar as in other familiar languages (lowest precedence first):

- ? : ternary operator

- || logical or

- && logical and

- ==, != equality and inequality

- IN in operator

- <, <=, >=, > less than, less equal, greater equal, greater than

- +, - addition, subtraction

- \*, /, % multiplication, division, modulus

- !, +, - logical negation, unary plus, unary minus

- [\*] expansion

- () function call

- . member access

- [] indexed value access

The parentheses ( and ) can be used to enforce a different operator evaluation order.

High-level operations

FOR

The FOR keyword can be to iterate over all elements of an array. The general syntax is:

FOR variable-name IN expression

There also is a special case for graph traversals:

FOR vertex-variable-name, edge-variable-name, path-variable-name IN traversal-expression

For this special case see the graph traversals chapter. For all other cases read on:

Each array element returned by expression is visited exactly once. It is required that expression returns an array in all cases. The empty array is allowed, too. The current array element is made available for further processing in the variable specified by variable-name.

FOR u IN users

RETURN u

This will iterate over all elements from the array users (note: this array consists of all documents from the collection named "users" in this case) and make the current array element available in variable u. u is not modified in this example but simply pushed into the result using the RETURN keyword.

Note: When iterating over collection-based arrays as shown here, the order of documents is undefined unless an explicit sort order is defined using a SORT statement.

The variable introduced by FOR is available until the scope the FOR is placed in is closed.

Another example that uses a statically declared array of values to iterate over:

FOR year IN [ 2011, 2012, 2013 ]

RETURN { "year" : year, "isLeapYear" : year % 4 == 0 && (year % 100 != 0 || year % 400 == 0) }

Nesting of multiple FOR statements is allowed, too. When FOR statements are nested, a cross product of the array elements returned by the individual FOR statements will be created.

FOR u IN users

FOR l IN locations

RETURN { "user" : u, "location" : l }

In this example, there are two array iterations: an outer iteration over the array users plus an inner iteration over the array locations. The inner array is traversed as many times as there are elements in the outer array. For each iteration, the current values of users and locations are made available for further processing in the variable u and l.

RETURN

The RETURN statement can be used to produce the result of a query. It is mandatory to specify a RETURN statement at the end of each block in a data-selection query, otherwise the query result would be undefined. Using RETURN on the main level in data-modification queries is optional.

The general syntax for RETURN is:

RETURN expression

The expression returned by RETURN is produced for each iteration in the block the RETURN statement is placed in. That means the result of a RETURN statement is always an array (this includes the empty array). To return all elements from the currently iterated array without modification, the following simple form can be used:

FOR variable-name IN expression

RETURN variable-name

As RETURN allows specifying an expression, arbitrary computations can be performed to calculate the result elements. Any of the variables valid in the scope the RETURN is placed in can be used for the computations.

Note: RETURN will close the current scope and eliminate all local variables in it.

RETURN DISTINCT

Since ArangoDB 2.7, RETURN can optionally be followed by the DISTINCT keyword. The DISTINCT keyword will ensure uniqueness of the values returned by the RETURN statement:

FOR variable-name IN expression

RETURN DISTINCT expression

If the DISTINCT is applied on an expression that itself is an array or a subquery, the DISTINCT will not make the values in each array or subquery result unique, but instead ensure that the result contains only distinct arrays or subquery results. To make the result of an array or a subquery unique, simply apply the DISTINCT for the array or the subquery.

For example, the following query will apply DISTINCT on its subquery results, but not inside the subquery:

FOR what IN 1..2

RETURN DISTINCT (

FOR i IN [ 1, 2, 3, 4, 1, 3 ]

RETURN i

)

Here we'll have a FOR loop with two iterations that each execute a subquery. The DISTINCT here is applied on the two subquery results. Both subqueries return the same result value (that is [ 1, 2, 3, 4, 1, 3 ]), so after DISTINCT there will only be one occurrence of the value [ 1, 2, 3, 4, 1, 3 ] left:

[

[ 1, 2, 3, 4, 1, 3 ]

]

If the goal is to apply the DISTINCT inside the subquery, it needs to be moved there:

FOR what IN 1..2

LET sub = (

FOR i IN [ 1, 2, 3, 4, 1, 3 ]

RETURN DISTINCT i

)

RETURN sub

In the above case, the DISTINCT will make the subquery results unique, so that each subquery will return a unique array of values ([ 1, 2, 3, 4 ]). As the subquery is executed twice and there is no DISTINCT on the top-level, that array will be returned twice:

[

[ 1, 2, 3, 4 ],

[ 1, 2, 3, 4 ]

]

Note: the order of results is undefined for RETURN DISTINCT.

Note: RETURN DISTINCT is not allowed on the top-level of a query if there is no FOR loop in front of it.

FILTER

The FILTER statement can be used to restrict the results to elements that match an arbitrary logical condition. The general syntax is:

FILTER condition

condition must be a condition that evaluates to either false or true. If the condition result is false, the current element is skipped, so it will not be processed further and not be part of the result. If the condition is true, the current element is not skipped and can be further processed.

FOR u IN users

FILTER u.active == true && u.age < 39

RETURN u

In the above example, all array elements from users will be included that have an attribute active with value true and that have an attribute age with a value less than 39 (including null ones). All other elements from users will be skipped and not be included the result produced by RETURN.

It is allowed to specify multiple FILTER statements in a query, and even in the same block. If multiple FILTER statements are used, their results will be combined with a logical and, meaning all filter conditions must be true to include an element.

FOR u IN users

FILTER u.active == true

FILTER u.age < 39

RETURN u

SORT

The SORT statement will force a sort of the array of already produced intermediate results in the current block. SORT allows specifying one or multiple sort criteria and directions. The general syntax is:

SORT expression direction

Specifying the direction is optional. The default (implicit) direction for a sort is the ascending order. To explicitly specify the sort direction, the keywords ASC (ascending) and DESC can be used. Multiple sort criteria can be separated using commas.

Note: when iterating over collection-based arrays, the order of documents is always undefined unless an explicit sort order is defined using SORT.

FOR u IN users

SORT u.lastName, u.firstName, u.id DESC

RETURN u

Note that constant SORT expressions can be used to indicate that no particular sort order is desired. Constant SORT expressions will be optimized away by the AQL optimizer during optimization, but specifying them explicitly may enable further optimizations if the optimizer does not need to take into account any particular sort order.

LIMIT

The LIMIT statement allows slicing the result array using an offset and a count. It reduces the number of elements in the result to at most the specified number. Two general forms of LIMIT are followed:

LIMIT count

LIMIT offset, count

The first form allows specifying only the count value whereas the second form allows specifying both offset and count. The first form is identical using the second form with an offset value of 0.

The offset value specifies how many elements from the result shall be discarded. It must be 0 or greater. The count value specifies how many elements should be at most included in the result.

FOR u IN users

SORT u.firstName, u.lastName, u.id DESC

LIMIT 0, 5

RETURN u

LET

The LET statement can be used to assign an arbitrary value to a variable. The variable is then introduced in the scope the LET statement is placed in. The general syntax is:

LET variable-name = expression

LET statements are mostly used to declare complex computations and to avoid repeated computations of the same value at multiple parts of a query.

FOR u IN users

LET numRecommendations = LENGTH(u.recommendations)

RETURN {

"user" : u,

"numRecommendations" : numRecommendations,

"isPowerUser" : numRecommendations >= 10

}

In the above example, the computation of the number of recommendations is factored out using a LET statement, thus avoiding computing the value twice in the RETURN statement.

Another use case for LET is to declare a complex computation in a subquery, making the whole query more readable.

FOR u IN users

LET friends = (

FOR f IN friends

FILTER u.id == f.userId

RETURN f

)

LET memberships = (

FOR m IN memberships

FILTER u.id == m.userId

RETURN m

)

RETURN {

"user" : u,

"friends" : friends,

"numFriends" : LENGTH(friends),

"memberShips" : memberships

}

COLLECT

The COLLECT keyword can be used to group an array by one or multiple group criteria.

The COLLECT statement will eliminate all local variables in the current scope. After COLLECT only the variables introduced by COLLECT itself are available.

The general syntaxes for COLLECT are:

COLLECT variable-name = expression options

COLLECT variable-name = expression INTO groups-variable options

COLLECT variable-name = expression INTO groups-variable = projection-expression options

COLLECT variable-name = expression INTO groups-variable KEEP keep-variable options

COLLECT variable-name = expression WITH COUNT INTO count-variable options

COLLECT variable-name = expression AGGREGATE variable-name = aggregate-expression options

COLLECT AGGREGATE variable-name = aggregate-expression options

COLLECT WITH COUNT INTO count-variable options

Grouping syntaxes

The first syntax form of COLLECT only groups the result by the defined group criteria specified in expression. In order to further process the results produced by COLLECT, a new variable (specified by variable-name) is introduced. This variable contains the group value.

Here's an example query that find the distinct values in u.city and makes them available in variable city:

FOR u IN users

COLLECT city = u.city

RETURN {

"city" : city

}

The second form does the same as the first form, but additionally introduces a variable (specified by groups-variable) that contains all elements that fell into the group. This works as follows: The groups-variable variable is an array containing as many elements as there are in the group. Each member of that array is a JSON object in which the value of every variable that is defined in the AQL query is bound to the corresponding attribute. Note that this considers all variables that are defined before the COLLECT statement, but not those on the top level (outside of any FOR), unless the COLLECT statement is itself on the top level, in which case all variables are taken. Furthermore note that it is possible that the optimizer moves LET statements out of FOR statements to improve performance.

FOR u IN users

COLLECT city = u.city INTO groups

RETURN {

"city" : city,

"usersInCity" : groups

}

In the above example, the array users will be grouped by the attribute city. The result is a new array of documents, with one element per distinct u.city value. The elements from the original array (here: users) per city are made available in the variable groups. This is due to the INTO clause.

COLLECT also allows specifying multiple group criteria. Individual group criteria can be separated by commas:

FOR u IN users

COLLECT country = u.country, city = u.city INTO groups

RETURN {

"country" : country,

"city" : city,

"usersInCity" : groups

}

In the above example, the array users is grouped by country first and then by city, and for each distinct combination of country and city, the users will be returned.

Discarding obsolete variables

The third form of COLLECT allows rewriting the contents of the groups-variable using an arbitrary projection-expression:

FOR u IN users

COLLECT country = u.country, city = u.city INTO groups = u.name

RETURN {

"country" : country,

"city" : city,

"userNames" : groups

}

In the above example, only the projection-expression is u.name. Therefore, only this attribute is copied into the groups-variable for each document. This is probably much more efficient than copying all variables from the scope into the groups-variable as it would happen without a projection-expression.

The expression following INTO can also be used for arbitrary computations:

FOR u IN users

COLLECT country = u.country, city = u.city INTO groups = {

"name" : u.name,

"isActive" : u.status == "active"

}

RETURN {

"country" : country,

"city" : city,

"usersInCity" : groups

}

COLLECT also provides an optional KEEP clause that can be used to control which variables will be copied into the variable created by  INTO . If no KEEP clause is specified, all variables from the scope will be copied as sub-attributes into the groups-variable. This is safe but can have a negative impact on performance if there are many variables in scope or the variables contain massive amounts of data.

The following example limits the variables that are copied into the groups-variable to just name. The variables u and someCalculation also present in the scope will not be copied into groups-variable because they are not listed in the KEEP clause:

FOR u IN users

LET name = u.name

LET someCalculation = u.value1 + u.value2

COLLECT city = u.city INTO groups KEEP name

RETURN {

"city" : city,

"userNames" : groups[*].name

}

KEEP is only valid in combination with INTO. Only valid variable names can be used in the KEEP clause. KEEP supports the specification of multiple variable names.

Group length calculation

COLLECT also provides a special WITH COUNT clause that can be used to determine the number of group members efficiently.

The simplest form just returns the number of items that made it into the COLLECT:

FOR u IN users

COLLECT WITH COUNT INTO length

RETURN length

The above is equivalent to, but more efficient than:

RETURN LENGTH(

FOR u IN users

RETURN length

)

The WITH COUNT clause can also be used to efficiently count the number of items in each group:

FOR u IN users

COLLECT age = u.age WITH COUNT INTO length

RETURN {

"age" : age,

"count" : length

}

Note: the WITH COUNT clause can only be used together with an INTO clause.

Aggregation

A  COLLECT  statement can be used to perform aggregation of data per group. To only determine group lengths, the  WITH COUNT INTO  variant of  COLLECT  can be used as described before.

For other aggregations, it is possible to run aggregate functions on the  COLLECT  results:

FOR u IN users

COLLECT ageGroup = FLOOR(u.age / 5) * 5 INTO g

RETURN {

"ageGroup" : ageGroup,

"minAge" : MIN(g[*].u.age),

"maxAge" : MAX(g[*].u.age)

}

The above however requires storing all group values during the collect operation for all groups, which can be inefficient.

The special  AGGREGATE  variant of  COLLECT  allows building the aggregate values incrementally during the collect operation, and is therefore often more efficient.

With the  AGGREGATE  variant the above query becomes:

FOR u IN users

COLLECT ageGroup = FLOOR(u.age / 5) * 5

AGGREGATE minAge = MIN(u.age), maxAge = MAX(u.age)

RETURN {

ageGroup,

minAge,

maxAge

}

The  AGGREGATE  keyword can only be used after the  COLLECT  keyword. If used, it must directly follow the declaration of the grouping keys. If no grouping keys are used, it must follow the  COLLECT  keyword directly:

FOR u IN users

COLLECT AGGREGATE minAge = MIN(u.age), maxAge = MAX(u.age)

RETURN {

minAge,

maxAge

}

Only specific expressions are allowed on the right-hand side of each  AGGREGATE  assignment:

- on the top level, an aggregate expression must be a call to one of the supported aggregation functions  LENGTH ,  MIN ,  MAX ,  SUM ,  AVERAGE ,  STDDEV_POPULATION ,  STDDEV_SAMPLE ,  VARIANCE_POPULATION , or  VARIANCE_SAMPLE

- an aggregate expression must not refer to variables introduced by the  COLLECT  itself

COLLECT variants

Since ArangoDB 2.6, there are two variants of COLLECT that the optimizer can choose from: the sorted variant and the hash variant. The hash variant only becomes a candidate for COLLECT statements that do not use an INTO clause.

The optimizer will always generate a plan that employs the sorted method. The sorted method requires its input to be sorted by the group criteria specified in the COLLECT clause. To ensure correctness of the result, the AQL optimizer will automatically insert a SORT statement into the query in front of the COLLECT statement. The optimizer may be able to optimize away that SORT statement later if a sorted index is present on the group criteria.

In case a COLLECT qualifies for using the hash variant, the optimizer will create an extra plan for it at the beginning of the planning phase. In this plan, no extra SORT statement will be added in front of the COLLECT. This is because the hash variant of COLLECT does not require sorted input. Instead, a SORT statement will be added after the COLLECT to sort its output. This SORT statement may be optimized away again in later stages. If the sort order of the COLLECT is irrelevant to the user, adding the extra instruction SORT null after the COLLECT will allow the optimizer to remove the sorts altogether:

FOR u IN users

COLLECT age = u.age

SORT null  /* note: will be optimized away */

RETURN age

Which COLLECT variant is used by the optimizer depends on the optimizer's cost estimations. The created plans with the different COLLECT variants will be shipped through the regular optimization pipeline. In the end, the optimizer will pick the plan with the lowest estimated total cost as usual.

In general, the sorted variant of COLLECT should be preferred in cases when there is a sorted index present on the group criteria. In this case the optimizer can eliminate the SORT statement in front of the COLLECT, so that no SORT will be left.

If there is no sorted index available on the group criteria, the up-front sort required by the sorted variant can be expensive. In this case it is likely that the optimizer will prefer the hash variant of COLLECT, which does not require its input to be sorted.

Which variant of COLLECT was actually used can be figured out by looking into the execution plan of a query, specifically the AggregateNode and its aggregationOptions attribute.

Setting COLLECT options

options can be used in a COLLECT statement to inform the optimizer about the preferred COLLECT method. When specifying the following appendix to a COLLECT statement, the optimizer will always use the sorted variant of COLLECT and not even create a plan using the hash variant:

OPTIONS { method: "sorted" }

Note that specifying hash as method will not make the optimizer use the hash variant. This is because the hash variant is not eligible for all queries. Instead, if no options or any other method than sorted are specified in OPTIONS, the optimizer will use its regular cost estimations.

COLLECT vs. RETURN DISTINCT

In order to make a result set unique, one can either use COLLECT or RETURN DISTINCT. Behind the scenes, both variants will work by creating an AggregateNode. For both variants, the optimizer may try the sorted and the hashed variant of COLLECT. The difference is therefore mainly syntactical, with RETURN DISTINCT saving a bit of typing when compared to an equivalent COLLECT:

FOR u IN users

RETURN DISTINCT u.age

FOR u IN users

COLLECT age = u.age

RETURN age

However, COLLECT is vastly more flexible than RETURN DISTINCT. Additionally, the order of results is undefined for a RETURN DISTINCT, whereas for a COLLECT the results will be sorted.

REMOVE

The REMOVE keyword can be used to remove documents from a collection. On a single server, the document removal is executed transactionally in an all-or-nothing fashion. For sharded collections, the entire remove operation is not transactional.

Each REMOVE operation is restricted to a single collection, and the collection name must not be dynamic. Only a single REMOVE statement per collection is allowed per AQL query, and it cannot be followed by read operations that access the same collection, by traversal operations, or AQL functions that can read documents.

The syntax for a remove operation is:

REMOVE key-expression IN collection options

collection must contain the name of the collection to remove the documents from. key-expression must be an expression that contains the document identification. This can either be a string (which must then contain the document key) or a document, which must contain a _key attribute.

The following queries are thus equivalent:

FOR u IN users

REMOVE { _key: u._key } IN users

FOR u IN users

REMOVE u._key IN users

FOR u IN users

REMOVE u IN users

Note: A remove operation can remove arbitrary documents, and the documents do not need to be identical to the ones produced by a preceding FOR statement:

FOR i IN 1..1000

REMOVE { _key: CONCAT('test', i) } IN users

FOR u IN users

FILTER u.active == false

REMOVE { _key: u._key } IN backup

Setting query options

options can be used to suppress query errors that may occur when trying to remove non-existing documents. For example, the following query will fail if one of the to-be-deleted documents does not exist:

FOR i IN 1..1000

REMOVE { _key: CONCAT('test', i) } IN users

By specifying the ignoreErrors query option, these errors can be suppressed so the query completes:

FOR i IN 1..1000

REMOVE { _key: CONCAT('test', i) } IN users OPTIONS { ignoreErrors: true }

To make sure data has been written to disk when a query returns, there is the waitForSync query option:

FOR i IN 1..1000

REMOVE { _key: CONCAT('test', i) } IN users OPTIONS { waitForSync: true }

Returning the removed documents

The removed documents can also be returned by the query. In this case, the  REMOVE  statement must be followed by a  RETURN  statement (intermediate  LET  statements are allowed, too). REMOVE  introduces the pseudo-value  OLD  to refer to the removed documents:

REMOVE key-expression IN collection options RETURN OLD

Following is an example using a variable named  removed  for capturing the removed documents. For each removed document, the document key will be returned.

FOR u IN users

REMOVE u IN users

LET removed = OLD

RETURN removed._key

UPDATE

The UPDATE keyword can be used to partially update documents in a collection. On a single server, updates are executed transactionally in an all-or-nothing fashion. For sharded collections, the entire update operation is not transactional.

Each UPDATE operation is restricted to a single collection, and the collection name must not be dynamic. Only a single UPDATE statement per collection is allowed per AQL query, and it cannot be followed by read operations that access the same collection, by traversal operations, or AQL functions that can read documents. The system attributes of documents (_key, _id, _from, _to, _rev) cannot be updated.

The two syntaxes for an update operation are:

UPDATE document IN collection options

UPDATE key-expression WITH document IN collection options

collection must contain the name of the collection in which the documents should be updated. document must be a document that contains the attributes and values to be updated. When using the first syntax, document must also contain the _key attribute to identify the document to be updated.

FOR u IN users

UPDATE { _key: u._key, name: CONCAT(u.firstName, u.lastName) } IN users

The following query is invalid because it does not contain a _key attribute and thus it is not possible to determine the documents to be updated:

FOR u IN users

UPDATE { name: CONCAT(u.firstName, u.lastName) } IN users

When using the second syntax, key-expression provides the document identification. This can either be a string (which must then contain the document key) or a document, which must contain a _key attribute.

The following queries are equivalent:

FOR u IN users

UPDATE u._key WITH { name: CONCAT(u.firstName, u.lastName) } IN users

FOR u IN users

UPDATE { _key: u._key } WITH { name: CONCAT(u.firstName, u.lastName) } IN users

FOR u IN users

UPDATE u WITH { name: CONCAT(u.firstName, u.lastName) } IN users

An update operation may update arbitrary documents which do not need to be identical to the ones produced by a preceding FOR statement:

FOR i IN 1..1000

UPDATE CONCAT('test', i) WITH { foobar: true } IN users

FOR u IN users

FILTER u.active == false

UPDATE u WITH { status: 'inactive' } IN backup

Setting query options

options can be used to suppress query errors that may occur when trying to update non-existing documents or violating unique key constraints:

FOR i IN 1..1000

UPDATE { _key: CONCAT('test', i) } WITH { foobar: true } IN users OPTIONS { ignoreErrors: true }

An update operation will only update the attributes specified in document and leave other attributes untouched. Internal attributes (such as _id, _key, _rev, _from and _to) cannot be updated and are ignored when specified in document. Updating a document will modify the document's revision number with a server-generated value.

When updating an attribute with a null value, ArangoDB will not remove the attribute from the document but store a null value for it. To get rid of attributes in an update operation, set them to null and provide the keepNull option:

FOR u IN users

UPDATE u WITH { foobar: true, notNeeded: null } IN users OPTIONS { keepNull: false }

The above query will remove the notNeeded attribute from the documents and update the foobar attribute normally.

There is also the option mergeObjects that controls whether object contents will be merged if an object attribute is present in both the UPDATE query and in the to-be-updated document.

The following query will set the updated document's name attribute to the exact same value that is specified in the query. This is due to the mergeObjects option being set to false:

FOR u IN users

UPDATE u WITH { name: { first: "foo", middle: "b.", last: "baz" } } IN users OPTIONS { mergeObjects: false }

Contrary, the following query will merge the contents of the name attribute in the original document with the value specified in the query:

FOR u IN users

UPDATE u WITH { name: { first: "foo", middle: "b.", last: "baz" } } IN users OPTIONS { mergeObjects: true }

Attributes in name that are present in the to-be-updated document but not in the query will now be preserved. Attributes that are present in both will be overwritten with the values specified in the query.

Note: the default value for mergeObjects is true, so there is no need to specify it explicitly.

To make sure data are durable when an update query returns, there is the waitForSync query option:

FOR u IN users

UPDATE u WITH { foobar: true } IN users OPTIONS { waitForSync: true }

Returning the modified documents

The modified documents can also be returned by the query. In this case, the  UPDATE  statement needs to be followed a  RETURN  statement (intermediate  LET  statements are allowed, too). These statements can refer to the pseudo-values  OLD  and  NEW . The  OLD  pseudo-value refers to the document revisions before the update, and  NEW  refers to document revisions after the update.

Both  OLD  and  NEW  will contain all document attributes, even those not specified in the update expression.

UPDATE document IN collection options RETURN OLD

UPDATE document IN collection options RETURN NEW

UPDATE key-expression WITH document IN collection options RETURN OLD

UPDATE key-expression WITH document IN collection options RETURN NEW

Following is an example using a variable named  previous  to capture the original documents before modification. For each modified document, the document key is returned.

FOR u IN users

UPDATE u WITH { value: "test" }

LET previous = OLD

RETURN previous._key

The following query uses the  NEW  pseudo-value to return the updated documents, without some of the system attributes:

FOR u IN users

UPDATE u WITH { value: "test" }

LET updated = NEW

RETURN UNSET(updated, "_key", "_id", "_rev")

It is also possible to return both  OLD  and  NEW :

FOR u IN users

UPDATE u WITH { value: "test" }

RETURN { before: OLD, after: NEW }

REPLACE

The REPLACE keyword can be used to completely replace documents in a collection. On a single server, the replace operation is executed transactionally in an all-or-nothing fashion. For sharded collections, the entire replace operation is not transactional.

Each REPLACE operation is restricted to a single collection, and the collection name must not be dynamic. Only a single REPLACE statement per collection is allowed per AQL query, and it cannot be followed by read operations that access the same collection, by traversal operations, or AQL functions that can read documents. The system attributes of documents (_key, _id, _from, _to, _rev) cannot be replaced.

The two syntaxes for a replace operation are:

REPLACE document IN collection options

REPLACE key-expression WITH document IN collection options

collection must contain the name of the collection in which the documents should be replaced. document is the replacement document. When using the first syntax, document must also contain the _key attribute to identify the document to be replaced.

FOR u IN users

REPLACE { _key: u._key, name: CONCAT(u.firstName, u.lastName), status: u.status } IN users

The following query is invalid because it does not contain a _key attribute and thus it is not possible to determine the documents to be replaced:

FOR u IN users

REPLACE { name: CONCAT(u.firstName, u.lastName, status: u.status) } IN users

When using the second syntax, key-expression provides the document identification. This can either be a string (which must then contain the document key) or a document, which must contain a _key attribute.

The following queries are equivalent:

FOR u IN users

REPLACE { _key: u._key, name: CONCAT(u.firstName, u.lastName) } IN users

FOR u IN users

REPLACE u._key WITH { name: CONCAT(u.firstName, u.lastName) } IN users

FOR u IN users

REPLACE { _key: u._key } WITH { name: CONCAT(u.firstName, u.lastName) } IN users

FOR u IN users

REPLACE u WITH { name: CONCAT(u.firstName, u.lastName) } IN users

A replace will fully replace an existing document, but it will not modify the values of internal attributes (such as _id, _key, _from and _to). Replacing a document will modify a document's revision number with a server-generated value.

A replace operation may update arbitrary documents which do not need to be identical to the ones produced by a preceding FOR statement:

FOR i IN 1..1000

REPLACE CONCAT('test', i) WITH { foobar: true } IN users

FOR u IN users

FILTER u.active == false

REPLACE u WITH { status: 'inactive', name: u.name } IN backup

Setting query options

options can be used to suppress query errors that may occur when trying to replace non-existing documents or when violating unique key constraints:

FOR i IN 1..1000

REPLACE { _key: CONCAT('test', i) } WITH { foobar: true } IN users OPTIONS { ignoreErrors: true }

To make sure data are durable when a replace query returns, there is the waitForSync query option:

FOR i IN 1..1000

REPLACE { _key: CONCAT('test', i) } WITH { foobar: true } IN users OPTIONS { waitForSync: true }

Returning the modified documents

The modified documents can also be returned by the query. In this case, the  REPLACE  statement must be followed by a  RETURN  statement (intermediate  LET  statements are allowed, too). The  OLD  pseudo-value can be used to refer to document revisions before the replace, and  NEW  refers to document revisions after the replace.

Both  OLD  and  NEW  will contain all document attributes, even those not specified in the replace expression.

REPLACE document IN collection options RETURN OLD

REPLACE document IN collection options RETURN NEW

REPLACE key-expression WITH document IN collection options RETURN OLD

REPLACE key-expression WITH document IN collection options RETURN NEW

Following is an example using a variable named  previous  to return the original documents before modification. For each replaced document, the document key will be returned:

FOR u IN users

REPLACE u WITH { value: "test" }

LET previous = OLD

RETURN previous._key

The following query uses the  NEW  pseudo-value to return the replaced documents (without some of their system attributes):

FOR u IN users

REPLACE u WITH { value: "test" }

LET replaced = NEW

RETURN UNSET(replaced, '_key', '_id', '_rev')

INSERT

The INSERT keyword can be used to insert new documents into a collection. On a single server, an insert operation is executed transactionally in an all-or-nothing fashion. For sharded collections, the entire insert operation is not transactional.

Each INSERT operation is restricted to a single collection, and the collection name must not be dynamic. Only a single INSERT statement per collection is allowed per AQL query, and it cannot be followed by read operations that access the same collection, by traversal operations, or AQL functions that can read documents.

The syntax for an insert operation is:

INSERT document IN collection options

Note: The INTO keyword is also allowed in the place of IN.

collection must contain the name of the collection into which the documents should be inserted. document is the document to be inserted, and it may or may not contain a _key attribute. If no _key attribute is provided, ArangoDB will auto-generate a value for _key value. Inserting a document will also auto-generate a document revision number for the document.

FOR i IN 1..100

INSERT { value: i } IN numbers

When inserting into an edge collection, it is mandatory to specify the attributes _from and _to in document:

FOR u IN users

FOR p IN products

FILTER u._key == p.recommendedBy

INSERT { _from: u._id, _to: p._id } IN recommendations

Setting query options

options can be used to suppress query errors that may occur when violating unique key constraints:

FOR i IN 1..1000

INSERT { _key: CONCAT('test', i), name: "test" } WITH { foobar: true } IN users OPTIONS { ignoreErrors: true }

To make sure data are durable when an insert query returns, there is the waitForSync query option:

FOR i IN 1..1000

INSERT { _key: CONCAT('test', i), name: "test" } WITH { foobar: true } IN users OPTIONS { waitForSync: true }

Returning the inserted documents

The inserted documents can also be returned by the query. In this case, the  INSERT  statement can be a  RETURN  statement (intermediate  LET  statements are allowed, too). To refer to the inserted documents, the  INSERT  statement introduces a pseudo-value named  NEW .

The documents contained in  NEW  will contain all attributes, even those auto-generated by the database (e.g.  _id ,  _key ,  _rev ,  _from , and  _to ).

INSERT document IN collection options RETURN NEW

Following is an example using a variable named  inserted  to return the inserted documents. For each inserted document, the document key is returned:

FOR i IN 1..100

INSERT { value: i }

LET inserted = NEW

RETURN inserted._key

UPSERT

The UPSERT keyword can be used for checking whether certain documents exist, and to update them in case they exist, or create them in case they do not exist. On a single server, upserts are executed transactionally in an all-or-nothing fashion. For sharded collections, the entire update operation is not transactional.

Each UPSERT operation is restricted to a single collection, and the collection name must not be dynamic. Only a single UPSERT statement per collection is allowed per AQL query, and it cannot be followed by read operations that access the same collection, by traversal operations, or AQL functions that can read documents.

The syntax for an upsert operation is:

UPSERT search-expression INSERT insert-expression UPDATE update-expression IN collection options

UPSERT search-expression INSERT insert-expression REPLACE update-expression IN collection options

When using the UPDATE variant of the upsert operation, the found document will be partially updated, meaning only the attributes specified in update-expression will be updated or added. When using the REPLACE variant of upsert, existing documents will be replaced with the contexts of update-expression.

Updating a document will modify the document's revision number with a server-generated value. The system attributes of documents (_key, _id, _from, _to, _rev) cannot be updated.

The search-expression contains the document to be looked for. It must be an object literal without dynamic attribute names. In case no such document can be found in collection, a new document will be inserted into the collection as specified in the insert-expression.

In case at least one document in collection matches the search-expression, it will be updated using the update-expression. When more than one document in the collection matches the search-expression, it is undefined which of the matching documents will be updated. It is therefore often sensible to make sure by other means (such as unique indexes, application logic etc.) that at most one document matches search-expression.

The following query will look in the users collection for a document with a specific name attribute value. If the document exists, its logins attribute will be increased by one. If it does not exist, a new document will be inserted, consisting of the attributes name, logins, and dateCreated:

UPSERT { name: 'superuser' }

INSERT { name: 'superuser', logins: 1, dateCreated: DATE_NOW() }

UPDATE { logins: OLD.logins + 1 } IN users

Note that in the UPDATE case it is possible to refer to the previous version of the document using the OLD pseudo-value.

Setting query options

As in several above examples, the ignoreErrors option can be used to suppress query errors that may occur when trying to violate unique key constraints.

When updating or replacing an attribute with a null value, ArangoDB will not remove the attribute from the document but store a null value for it. To get rid of attributes in an upsert operation, set them to null and provide the keepNull option.

There is also the option mergeObjects that controls whether object contents will be merged if an object attribute is present in both the UPDATE query and in the to-be-updated document.

Note: the default value for mergeObjects is true, so there is no need to specify it explicitly.

To make sure data are durable when an update query returns, there is the waitForSync query option.

Returning documents

UPSERT  statements can optionally return data. To do so, they need to be followed by a  RETURN  statement (intermediate  LET  statements are allowed, too). These statements can optionally perform calculations and refer to the pseudo-values  OLD  and  NEW . In case the upsert performed an insert operation,  OLD  will have a value of null. In case the upsert performed an update or replace operation,  OLD  will contain the previous version of the document, before update/replace.

NEW  will always be populated. It will contain the inserted document in case the upsert performed an insert, or the updated/replaced document in case it performed an update/replace.

This can also be used to check whether the upsert has performed an insert or an update internally:

UPSERT { name: 'superuser' }

INSERT { name: 'superuser', logins: 1, dateCreated: DATE_NOW() }

UPDATE { logins: OLD.logins + 1 } IN users

RETURN { doc: NEW, type: OLD ? 'update' : 'insert' }

Graphs in AQL

As you already read about graphs in ArangoDB you can have several views on graphs. There are also several ways to work with graphs in AQL. You can use named graphs where ArangoDB manages the collections involved in one graph. You can also use graph functions on a combination of document and edge collections. named graphs are defined though the graph-module, that contains the name of the graph, and the vertex and edge collections involved. Since the management functions are layered on top of simple sets of document and edge collections, you can also use regular AQL functions to work with them.

In AQL you can reach several graphing functions:

- AQL Traversals is making full use of optimisations and therefore best performance is to be expected. It can work on named graphs and loosely coupled collection sets (aka anonymous graphs). You can use AQL filter conditions on traversals.

- Named graph Operations work on named graphs; offer a versatile range of parameters.

- Other graph functions work on single edge collection (which may also be part of named graphs).

Graph traversals in AQL

General query idea

This query is only useful if you use edge collections and/or graphs in your data model. It is supposed to walk through your graph. Therefore it starts at one specific document (start-vertex) and follows all edges connected to this document. For all documents (vertices) that are targeted by these edges it will again follow all edges connected to them and so on. It is possible to define how many of these follow iterations should be executed at least (min-depth) and at most (max-depth). For all vertices that where visited during this process in the range between min-depth and max-depth iterations you will get a result in form of a set with three items:

1. The visited vertex.

2. The edge pointing to it

3. The complete path from start-vertex to the visited vertex as a JSON document with an attribute  edges  and an attribute  vertices , each a list of the coresponding elements. These lists are sorted, s.t. the first element in  vertices  is the start-vertex and the last is the visited vertex. And that the n-th element in  edges  connects the n-th element with the (n+1)-th element in  vertices .

Example execution

Let's take a look at a simple example to explain how it works. This is the graph that we are going to traverse:

Now we use the following parameters for our query:

1. We start at the vertex A.

2. We use a min-depth of 1.

3. We use a max-depth of 2.

4. We follow only outbound direction of edges

Now it walks to one of the direct neighbors of A, say B (NOTE: ordering is not guaranteed):

The query will remember the state (red circle) and will emit the first result A -> B (black box). This will also prevent the traverser to be trapped in cycles. Now again it will visit one of the direct neighbors of B, say E:

We have limited the query with a max-depth of 2 then it will not pick any neighbor of E as the path from A to E already requires 2 steps. Instead we will go back one level to B and continue with any other direct neighbor there:

Again after we produced this result we will step back to B. But there is no neighbor of B left that we have not yet visited. Hence we go another step back to A and continue with any other neighbor there.

And identical to the iterations before we will visit H:

And J:

And after these steps there is no further result left. So all together this query has returned the following paths:

1. A -> B

2. A -> B -> E

3. A -> B -> C

4. A -> G

5. A -> G -> H

6. A -> G -> J

Syntax

Now let's see how we can write a query that follows this schema. You have two options here, you can either use a named graph (see the graphs chapter on how to create it) or anonymous graphs.

Working on named graphs:

FOR  vertex[, edge[, path]]  IN   MIN [.. MAX ]  OUTBOUND|INBOUND|ANY  startVertex  GRAPH  graphName

- FOR  - emits up to three variables:

- vertex: the current vertex in a traversal

- edge: (optional) the current edge in a traversal

- path: (optional, requires edge to be present) an object representing the current path with two members:

- vertices : an array of all vertices on this path.

- edges : an array of all edges on this path.

- IN   MIN .. MAX   OUTBOUND  startVertex  GRAPH  graphName

- OUTBOUND|INBOUND|ANY  traversal will be done for outbound / inbound / inbound+outbound pointing edges

- startVertex: one vertex where the traversal will originate from, this can be specified in the form of an id string or in the form of a document with the attribute  _id . All other values will lead to a warning and an empty result. If the specified id does not exist, the result is empty as well and there is no warning.

- graphName: the name identifying the named graph. It's vertex and edge collections will be looked up.

- MIN : edges and vertices returned by this query will start at the traversal depth of  MIN  (thus edges and vertices below will not be returned). If not specified, defaults to  1 , which is the minimal possible value.

- MAX : up to  MAX  length paths are traversed. If omitted in the query,  MAX  equals  MIN . Thus only the vertices and edges in the range of  MIN  are returned.

Working on collection sets:

FOR  vertex[, edge[, path]]  IN   MIN [.. MAX ]  OUTBOUND|INBOUND|ANY  startVertex edgeCollection1, .., edgeCollectionN

Instead of the  GRAPH graphName  you may specify a list of edge collections. Vertex collections are evaluated from the edges. The rest of the behavior is similar to the named version.

Working on collection sets with mixed directions:

For traversals with a list of edge collections you can optionally specify the direction for some of the edge collections. Say for example you have three edge collections  edges1 ,  edges2  and  edges3 , where in  edges2  the direction has no relevance but in  edges1  and  edges3  the direction should be taken into account. In this case you can use  OUTBOUND  as general traversal direction and  ANY  specifically for  edges2  in the following statement:

FOR vertex IN OUTBOUND startVertex edges1, ANY edges2, edges3

All collections in the list that do not specify their own direction will use the direction defined after  IN . This allows to use a different direction for each collection in your traversal.

Using filters and the explainer to extrapolate the costs

All three variables emitted by the traversals might as well be used in filter statements. For some of these filter statements the optimizer can detect that it is possible to prune paths of traversals earlier, hence filtered results will not be emitted to the variables in the first place. This may significantly improve the performance of your query. Whenever a filter is not fulfilled the complete set of vertex, edge and path will be skipped. All paths with a length greater than  MAX  will never be computed.

In the current state  OR  combined filters cannot be optimized,  AND  combined filters can.

Filtering on paths

This allows for the most powerful filtering and may have the highest impact on performance. Using the path variable you can filter on specific iteration depths. You can filter for absolute positions in the path by (specifying a positive number) (which then qualifies for the optimizations) or relative positions to the end of the path by specifying a negative number. Note: In the current state there is no way to define a filter for all elements on the path. This will be added in the future.

Filtering edges on the path

FOR v, e, p IN 1..5 OUTBOUND 'circles/A' GRAPH 'traversalGraph'

FILTER p.edges[0].theTruth == true

RETURN p

will filter all paths where the start edge (# 0) has the attribute  theTruth  equaling  true . The resulting paths will be up to 5 items long.

Filtering vertices on the path

Similar to filtering the edges on the path you can also filter the vertices:

FOR v, e, p IN 1..5 OUTBOUND 'circles/A' GRAPH 'traversalGraph'

FILTER p.vertices[1]._key == "G"

RETURN p

Combining several filters

And of course you can combine these filters in any way you like:

FOR v, e, p IN 1..5 OUTBOUND 'circles/A' GRAPH 'traversalGraph'

FILTER p.edges[0].theTruth == true

AND p.edges[1].theFalse == false

FILTER p.vertices[1]._key == "G"

RETURN p

will filter all paths where the first edge has the attribute  theTruth  equaling  true , the first vertex is "G" and the second edge has the attribute  theFalse  equaling  false . The resulting paths will be up to 5 items long. Note here: Although we have defined a  MIN  of  1  we will only get results of depth  2 . This is because for all results in depth  1  the second edge does not exist and hence cannot fulfill the condition.

Examples

We will create a simple symmetric traversal demonstration graph:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("traversalGraph");

arangosh> db.circles.toArray();

arangosh> db.edges.toArray();

show execution results

To get started we select the full graph; for better overview we only return the vertex ids:

arangosh> db._query("FOR v IN 1..3 OUTBOUND 'circles/A' GRAPH 'traversalGraph' RETURN v._key");

arangosh> db._query("FOR v IN 1..3 OUTBOUND 'circles/A' edges RETURN v._key");

show execution results

We can nicely see its heading for the first outer vertex, then going back to the branch to descend into the next tree. After that it returns to our start node, to descend again. As we can see both queries return the same result, the first one uses the named graph, the second directly uses the edge collection.

Now we only want the elements of a specific depth - 2 - the ones that are right behind the fork:

arangosh> db._query("FOR v IN 2..2 OUTBOUND 'circles/A' GRAPH 'traversalGraph' return v._key");

arangosh> db._query("FOR v IN 2 OUTBOUND 'circles/A' GRAPH 'traversalGraph' return v._key");

show execution results

As you can see, we can express this in two ways, one is to omit the  MAX  parameter of the expression.

Filter examples

Now lets start to add some filters. We want to cut of the branch on the right side of th graph, we may filter in two ways:

- we know the vertex at depth 1 has  _key  ==  G

- we know the  label  attribute of the edge connecting A to G is  right_foo

arangosh> db._query("FOR v, e, p IN 1..3 OUTBOUND 'circles/A' GRAPH 'traversalGraph' FILTER p.vertices[1]._key != 'G' RETURN v._key");

arangosh> db._query("FOR v, e, p IN 1..3 OUTBOUND 'circles/A' GRAPH 'traversalGraph' FILTER p.edges[0].label != 'right_foo' RETURN v._key");

show execution results

As we can see all vertices behind G are skipped in both queries. The first filters on the vertex  _key , the second on an edge label. Note again as soon as a filter is not fulfilled for any of the three elements  v ,  e  or  p  the complete set of these will be excluded from the result.

We also may combine several filters, for instance to filter out the right branch (G), and the E branch:

arangosh> db._query("FOR v,e,p IN 1..3 OUTBOUND 'circles/A' GRAPH 'traversalGraph' FILTER p.vertices[1]._key != 'G' FILTER p.edges[1].label != 'left_blub' return v._key");

arangosh> db._query("FOR v,e,p IN 1..3 OUTBOUND 'circles/A' GRAPH 'traversalGraph' FILTER p.vertices[1]._key != 'G' AND    p.edges[1].label != 'left_blub' return v._key");

show execution results

As you can see combining two  FILTER  statements with an  AND  has the same result.

Comparing OUTBOUND / INBOUND / ANY

All our previous examples traversed the graph into outbound edge directions. You may however want to also traverse in reverse direction ( INBOUND ) or both ( ANY ), Since  circles/A  only has outbound edges, we start our queries from  circles/E :

arangosh> db._query("FOR v IN 1..3 OUTBOUND 'circles/E' GRAPH 'traversalGraph' return v._key");

arangosh> db._query("FOR v IN 1..3 INBOUND 'circles/E' GRAPH 'traversalGraph' return v._key");

arangosh> db._query("FOR v IN 1..3 ANY 'circles/E' GRAPH 'traversalGraph' return v._key");

show execution results

The first traversal will only walk into the forward ( OUTBOUND ) direction. Therefore from E we only can see F. Walking into reverse direction ( INBOUND ) we see the path to A: B, A. Walking in forward and reverse direction ( ANY ) we can see a more diverse result. First of all we see the simple paths to F and A. However these vertices have edges in other directions and they will be traversed. Note here: The traverser may use identical edges multiple times. For instance if it walks from E to F it will continue to walk from F to E using the same edge once again. Due to this we will see duplicate nodes in the result.

Use the AQL explainer for optimizations

Now lets have a look what the optimizer does behind the curtains and inspect traversal queries using the explainer:

arangosh> db._explain("FOR v,e,p IN 1..3 OUTBOUND 'circles/A' GRAPH 'traversalGraph' LET localScopeVar = RAND() > 0.5 FILTER p.edges[0].theTruth != localScopeVar RETURN v._key", {}, {colors: false});

arangosh> db._explain("FOR v,e,p IN 1..3 OUTBOUND 'circles/A' GRAPH 'traversalGraph' FILTER p.edges[0].label == 'right_foo' RETURN v._key", {}, {colors: false});

show execution results

We now see two queries, in one we add a variable  localScopeVar  which is outside the scope of the traversal itself - it is not known inside of the traverser. Therefore this filter can only executed after the traversal, which may be undesired in large graphs. The second query on the other hand only operates on the path, and therefore this condition can be used during the execution of the traversal, paths that are filtered out by this condition won't be processed at all.

And finally clean it up again:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> examples.dropGraph("traversalGraph");

true

If this traversal is not powerful enough for your needs, so you cannot describe your conditions as AQL filter statements you might want to look at manually crafted traverser.

See here for more traversal examples.

Graph operations

This chapter describes graph related AQL functions.

Short explanation of the example parameter

A lot of the following functions accept a vertex (or edge) example as parameter. This can contain the following:

- {} : Returns all possible vertices for this graph

- idString : Returns the vertex/edge with the id idString

- [idString1, idString2 ...] : Returns the vertices/edges with the ids matching the given strings.

- {key1 : value1, key2 : value2} : Returns the vertices/edges that match this example, which means that both have key1 and key2 with the corresponding attributes

- {key1.key2 : value1, key3 : value2} : It is possible to chain keys, which means that a document {key1 : {key2 : value1}, key3 : value2} would be a match

- [{key1 : value1}, {key2 : value2}] : Returns the vertices/edges that match one of the examples, which means that either key1 or key2 are set with the corresponding value

The complexity of the shortest path algorithms

Most of the functions described in this chapter calculate the shortest paths for subsets of the graphs vertices. Hence the complexity of these functions depends of the chosen algorithm for this task. For Floyd-Warshall it is O(n^3) with n being the amount of vertices in the graph. For Dijkstra it would be O(x*y*n^2) with n being the amount of vertices in the graph, x the amount of start vertices and y the amount of target vertices. Hence a suggestion may be to use Dijkstra when x*y < n and the functions supports choosing your algorithm.

Example Graph

All examples in this chapter will use this simple city graph:

Edges and Vertices related functions

This section describes various AQL functions which can be used to receive information about the graph's vertices, edges, neighbor relationship and shared properties.

GRAPH_EDGES

GRAPH_EDGES (graphName, vertexExample, options)

The GRAPH_EDGES function returns all edges of the graph connected to the vertices defined by the example.

The complexity of this method is O(n*m^x) with n being the vertices defined by the parameter vertexExamplex, m the average amount of edges of a vertex and x the maximal depths. Hence the default call would have a complexity of O(n*m);

Parameters

- graphName : The name of the graph as a string.

- vertexExample : An example for the desired vertices (see example).

- options (optional) : An object containing the following options:

- direction : The direction of the edges as a string. Possible values are outbound, inbound and any (default).

- edgeCollectionRestriction : One or multiple edge collection names. Only edges from these collections will be considered for the path.

- startVertexCollectionRestriction : One or multiple vertex collection names. Only vertices from these collections will be considered as start vertex of a path.

- endVertexCollectionRestriction : One or multiple vertex collection names. Only vertices from these collections will be considered as end vertex of a path.

- edgeExamples : A filter example for the edges (see example).

- minDepth : Defines the minimal length of a path from an edge to a vertex (default is 1, which means only the edges directly connected to a vertex would be returned).

- maxDepth : Defines the maximal length of a path from an edge to a vertex (default is 1, which means only the edges directly connected to a vertex would be returned).

- maxIterations: the maximum number of iterations that the traversal is allowed to perform. It is sensible to set this number so unbounded traversals will terminate.

- includeData: Defines if the result should contain only ids (false) or if all documents should be fully extracted (true). By default this parameter is set to false, so only ids will be returned.

Examples

A route planner example, all edges to locations with a distance of either 700 or 600.:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_EDGES("

........> +"'routeplanner', {}, {edgeExamples : [{distance: 600}, {distance: 700}]}) RETURN e"

........> ).toArray();

[

"internationalHighway/1337913576"

]

A route planner example, all outbound edges of Hamburg with a maximal depth of 2 :

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_EDGES("

........> +"'routeplanner', 'germanCity/Hamburg', {direction : 'outbound', maxDepth : 2}) RETURN e"

........> ).toArray();

show execution results

Including the data:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_EDGES("

........> + "'routeplanner', 'germanCity/Hamburg', {direction : 'outbound',"

........> + "maxDepth : 2, includeData: true}) RETURN e"

........> ).toArray();

show execution results

GRAPH_VERTICES

The GRAPH_VERTICES function returns all vertices.

GRAPH_VERTICES (graphName, vertexExample, options)

According to the optional filters it will only return vertices that have outbound, inbound or any (default) edges.

Parameters

- graphName : The name of the graph as a string.

- vertexExample : An example for the desired vertices (see example).

- options (optional) : An object containing the following options:

- direction : The direction of the edges as a string. Possible values are outbound, inbound and any (default).

- vertexCollectionRestriction : One or multiple vertex collections that should be considered.

Examples

A route planner example, all vertices of the graph

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_VERTICES("

........> +"'routeplanner', {}) RETURN e").toArray();

show execution results

A route planner example, all vertices from collection germanCity.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_VERTICES("

........> +"'routeplanner', {}, {direction : 'any', vertexCollectionRestriction" +

........> " : 'germanCity'}) RETURN e").toArray();

show execution results

GRAPH_NEIGHBORS

The GRAPH_NEIGHBORS function returns all neighbors of vertices.

GRAPH_NEIGHBORS (graphName, vertexExample, options)

By default only the direct neighbors (path length equals 1) are returned, but one can define the range of the path length to the neighbors with the options minDepth and maxDepth. The complexity of this method is O(n*m^x) with n being the vertices defined by the parameter vertexExamplex, m the average amount of neighbors and x the maximal depths. Hence the default call would have a complexity of O(n*m);

Parameters

- graphName : The name of the graph as a string.

- vertexExample : An example for the desired vertices (see example).

- options : An object containing the following options:

- direction : The direction of the edges. Possible values are outbound, inbound and any (default).

- edgeExamples : A filter example for the edges to the neighbors (see example).

- neighborExamples : An example for the desired neighbors (see example).

- edgeCollectionRestriction : One or multiple edge collection names. Only edges from these collections will be considered for the path.

- vertexCollectionRestriction : One or multiple vertex collection names. Only vertices from these collections will be contained in the result. This does not effect vertices on the path.

- minDepth : Defines the minimal depth a path to a neighbor must have to be returned (default is 1).

- maxDepth : Defines the maximal depth a path to a neighbor must have to be returned (default is 1).

- maxIterations: the maximum number of iterations that the traversal is allowed to perform. It is sensible to set this number so unbounded traversals will terminate at some point.

- includeData is a boolean value to define if the returned documents should be extracted instead of returning their ids only. The default is false.

Note: in ArangoDB versions prior to 2.6 NEIGHBORS returned the array of neighbor vertices with all attributes and not just the vertex ids. To return to the same behavior, set the includeData option to true in 2.6 and above.

Examples

A route planner example, all neighbors of locations with a distance of either 700 or 600.:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_NEIGHBORS("

........> +"'routeplanner', {}, {edgeExamples : [{distance: 600}, {distance: 700}]}) RETURN e"

........> ).toArray();

[

"germanCity/Cologne",

"frenchCity/Lyon"

]

A route planner example, all outbound neighbors of Hamburg with a maximal depth of 2 :

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_NEIGHBORS("

........> +"'routeplanner', 'germanCity/Hamburg', {direction : 'outbound', maxDepth : 2}) RETURN e"

........> ).toArray();

show execution results

GRAPH_COMMON_NEIGHBORS

The GRAPH_COMMON_NEIGHBORS function returns all common neighbors of the vertices defined by the examples.

GRAPH_COMMON_NEIGHBORS (graphName, vertex1Example, vertex2Examples, optionsVertex1, optionsVertex2)

This function returns the intersection of GRAPH_NEIGHBORS(vertex1Example, optionsVertex1) and GRAPH_NEIGHBORS(vertex2Example, optionsVertex2). The complexity of this method is O(n*m^x) with n being the maximal amount of vertices defined by the parameters vertexExamples, m the average amount of neighbors and x the maximal depths. Hence the default call would have a complexity of O(n*m);

For parameter documentation read the documentation of GRAPH_NEIGHBORS.

Examples

A route planner example, all common neighbors of capitals.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_COMMON_NEIGHBORS("

........> +"'routeplanner', {isCapital : true}, {isCapital : true}) RETURN e"

........> ).toArray();

show execution results

A route planner example, all common outbound neighbors of Hamburg with any other location which have a maximal depth of 2:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_COMMON_NEIGHBORS("

........> +"'routeplanner', 'germanCity/Hamburg', {}, {direction : 'outbound', maxDepth : 2}, "+

........> "{direction : 'outbound', maxDepth : 2}) RETURN e"

........> ).toArray();

show execution results

GRAPH_COMMON_PROPERTIES

GRAPH_COMMON_PROPERTIES (graphName, vertex1Example, vertex2Examples, options)

The GRAPH_COMMON_PROPERTIES function returns a list of objects which have the id of the vertices defined by vertex1Example as keys and a list of vertices defined by vertex21Example, that share common properties as value. Notice that only the vertex id and the matching attributes are returned in the result.

The complexity of this method is O(n) with n being the maximal amount of vertices defined by the parameters vertexExamples.

Parameters

- graphName : The name of the graph as a string.

- vertex1Example : An example for the desired vertices (see example).

- vertex2Example : An example for the desired vertices (see example).

- options (optional) : An object containing the following options:

- vertex1CollectionRestriction : One or multiple vertex collection names. Only vertices from these collections will be considered.

- vertex2CollectionRestriction : One or multiple vertex collection names. Only vertices from these collections will be considered.

- ignoreProperties : One or multiple attributes of a document that should be ignored, either a string or an array..

Examples

A route planner example, all locations with the same properties:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_COMMON_PROPERTIES("

........> +"'routeplanner', {}, {}) RETURN e"

........> ).toArray();

show execution results

A route planner example, all cities which share same properties except for population.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_COMMON_PROPERTIES("

........> +"'routeplanner', {}, {}, {ignoreProperties: 'population'}) RETURN e"

........> ).toArray();

show execution results

Shortest Paths, distances and traversals.

This section describes AQL functions, that calculate paths from a subset of vertices in a graph to another subset of vertices.

GRAPH_PATHS

The GRAPH_PATHS function returns all paths of a graph.

GRAPH_PATHS (graphName, options)

The complexity of this method is O(n*n*m) with n being the amount of vertices in the graph and m the average amount of connected edges;

Parameters

- graphName : The name of the graph as a string.

- options : An object containing the following options:

- direction : The direction of the edges. Possible values are any, inbound and outbound (default).

- followCycles (optional) : If set to true the query follows cycles in the graph, default is false.

- minLength (optional) : Defines the minimal length a path must have to be returned (default is 0).

- maxLength (optional) : Defines the maximal length a path must have to be returned (default is 10).

Examples

Return all paths of the graph "social":

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("social");

arangosh> db._query("RETURN GRAPH_PATHS('social')").toArray();

show execution results

Return all inbound paths of the graph "social" with a maximal length of 1 and a minimal length of 2:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("social");

arangosh> db._query(

........> "RETURN GRAPH_PATHS('social', {direction : 'inbound', minLength : 1, maxLength :  2})"

........> ).toArray();

show execution results

GRAPH_SHORTEST_PATH

The GRAPH_SHORTEST_PATH function returns all shortest paths of a graph.

GRAPH_SHORTEST_PATH (graphName, startVertexExample, endVertexExample, options)

This function determines all shortest paths in a graph identified by graphName. If one wants to call this function to receive nearly all shortest paths for a graph the option algorithm should be set to Floyd-Warshall to increase performance. If no algorithm is provided in the options the function chooses the appropriate one (either Floyd-Warshall or Dijkstra) according to its parameters. The length of a path is by default the amount of edges from one start vertex to an end vertex. The option weight allows the user to define an edge attribute representing the length.

The complexity of the function is described here.

Parameters

- graphName : The name of the graph as a string.

- startVertexExample : An example for the desired start Vertices (see example).

- endVertexExample : An example for the desired end Vertices (see example).

- options (optional) : An object containing the following options:

- direction : The direction of the edges as a string. Possible values are outbound, inbound and any (default).

- edgeCollectionRestriction : One or multiple edge collection names. Only edges from these collections will be considered for the path.

- startVertexCollectionRestriction : One or multiple vertex collection names. Only vertices from these collections will be considered as start vertex of a path.

- endVertexCollectionRestriction : One or multiple vertex collection names. Only vertices from these collections will be considered as end vertex of a path.

- edgeExamples : A filter example for the edges in the shortest paths (see example).

- algorithm : The algorithm to calculate the shortest paths. If both start and end vertex examples are empty Floyd-Warshall is used, otherwise the default is Dijkstra.

- weight : The name of the attribute of the edges containing the length as a string.

- defaultWeight : Only used with the option weight. If an edge does not have the attribute named as defined in option weight this default is used as length. If no default is supplied the default would be positive Infinity so the path could not be calculated.

- stopAtFirstMatch : Only useful if targetVertices is an example that matches to more than one vertex. If so only the shortest path to the closest of these target vertices is returned. This flag is of special use if you have target pattern and you want to know which vertex with this pattern is matched first.

- includeData : Triggers if only _id's are returned (false, default) or if data is included for all objects as well (true) This will modify the content of vertex, path.vertices and path.edges.

NOTE: Since version 2.6 we have included a new optional parameter includeData. This parameter triggers if the result contains the real data object true or it just includes the _id values false. The default value is false as it yields performance benefits.

Examples

A route planner example, shortest distance from all german to all french cities:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_SHORTEST_PATH("

........> + "'routeplanner', {}, {}, {" +

........> "weight : 'distance', endVertexCollectionRestriction : 'frenchCity', " +

........> "includeData: true, " +

........> "startVertexCollectionRestriction : 'germanCity'}) RETURN [e.startVertex, e.vertex._id, " +

........> "e.distance, LENGTH(e.paths)]"

........> ).toArray();

show execution results

A route planner example, shortest distance from Hamburg and Cologne to Lyon:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_SHORTEST_PATH("

........> +"'routeplanner', [{_id: 'germanCity/Cologne'},{_id: 'germanCity/Munich'}], " +

........> "'frenchCity/Lyon', " +

........> "{weight : 'distance'}) RETURN [e.startVertex, e.vertex, e.distance, LENGTH(e.paths)]"

........> ).toArray();

show execution results

GRAPH_TRAVERSAL

The GRAPH_TRAVERSAL function traverses through the graph.

GRAPH_TRAVERSAL (graphName, startVertexExample, direction, options)

This function performs traversals on the given graph.

The complexity of this function strongly depends on the usage.

Parameters

- graphName : The name of the graph as a string.

- startVertexExample : An example for the desired vertices (see example).

- direction : The direction of the edges as a string. Possible values are outbound, inbound and any (default).

- options: Object containing optional options.

Options:

- strategy: determines the visitation strategy. Possible values are depthfirst and breadthfirst. Default is breadthfirst.

- order: determines the visitation order. Possible values are preorder and postorder.

- itemOrder: determines the order in which connections returned by the expander will be processed. Possible values are forward and backward.

- maxDepth: if set to a value greater than 0, this will limit the traversal to this maximum depth.

- minDepth: if set to a value greater than 0, all vertices found on a level below the minDepth level will not be included in the result.

- maxIterations: the maximum number of iterations that the traversal is allowed to perform. It is sensible to set this number so unbounded traversals will terminate at some point.

- uniqueness: an object that defines how repeated visitations of vertices should be handled. The uniqueness object can have a sub-attribute vertices, and a sub-attribute edges. Each sub-attribute can have one of the following values:

- "none": no uniqueness constraints

- "path": element is excluded if it is already contained in the current path. This setting may be sensible for graphs that contain cycles (e.g. A -> B -> C -> A).

- "global": element is excluded if it was already found/visited at any point during the traversal.

- filterVertices An optional array of example vertex documents that the traversal will treat specially. If no examples are given, the traversal will handle all encountered vertices equally. If one or many vertex examples are given, the traversal will exclude any non-matching vertex from the result and/or not descend into it. Optionally, filterVertices can contain a string containing the name of a user-defined AQL function that should be responsible for filtering. If so, the AQL function is expected to have the following signature:

function (config, vertex, path)

If a custom AQL function is used for filterVertices, it is expected to return one of the following values:

- [ ]: Include the vertex in the result and descend into its connected edges

- [ "prune" ]: Will include the vertex in the result but not descend into its connected edges

- [ "exclude" ]: Will not include the vertex in the result but descend into its connected edges

- [ "prune", "exclude" ]: Will completely ignore the vertex and its connected edges

- vertexFilterMethod: Only useful in conjunction with filterVertices and if no user-defined AQL function is used. If specified, it will influence how vertices are handled that don't match the examples in filterVertices:

- [ "prune" ]: Will include non-matching vertices in the result but not descend into them

- [ "exclude" ]: Will not include non-matching vertices in the result but descend into them

- [ "prune", "exclude" ]: Will completely ignore the vertex and its connected edges

Examples

A route planner example, start a traversal from Hamburg :

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_TRAVERSAL('routeplanner', 'germanCity/Hamburg'," +

........> " 'outbound') RETURN e"

........> ).toArray();

show execution results

A route planner example, start a traversal from Hamburg with a max depth of 1 so only the direct neighbors of Hamburg are returned:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_TRAVERSAL('routeplanner', 'germanCity/Hamburg'," +

........> " 'outbound', {maxDepth : 1}) RETURN e"

........> ).toArray();

show execution results

GRAPH_TRAVERSAL_TREE

The GRAPH_TRAVERSAL_TREE function traverses through the graph.

GRAPH_TRAVERSAL_TREE (graphName, startVertexExample, direction, connectName, options)  This function creates a tree format from the result for a better visualization of the path. This function performs traversals on the given graph.

The complexity of this function strongly depends on the usage.

Parameters

- graphName : The name of the graph as a string.

- startVertexExample : An example for the desired vertices (see example).

- direction : The direction of the edges as a string. Possible values are outbound, inbound and any (default).

- connectName : The result attribute which contains the connection.

- options (optional) : An object containing options, see Graph Traversals:

Examples

A route planner example, start a traversal from Hamburg :

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_TRAVERSAL_TREE('routeplanner', 'germanCity/Hamburg'," +

........> " 'outbound', 'connnection') RETURN e"

........> ).toArray();

show execution results

A route planner example, start a traversal from Hamburg with a max depth of 1 so only the direct neighbors of Hamburg are returned:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_TRAVERSAL_TREE('routeplanner', 'germanCity/Hamburg',"+

........> " 'outbound', 'connnection', {maxDepth : 1}) RETURN e"

........> ).toArray();

show execution results

GRAPH_DISTANCE_TO

The GRAPH_DISTANCE_TO function returns all paths and there distance within a graph.

GRAPH_DISTANCE_TO (graphName, startVertexExample, endVertexExample, options)

This function is a wrapper of GRAPH_SHORTEST_PATH. It does not return the actual path but only the distance between two vertices.

Examples

A route planner example, distance from all french to all german cities:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_DISTANCE_TO("

........> +"'routeplanner', {}, {}, { " +

........> " weight : 'distance', endVertexCollectionRestriction : 'germanCity', " +

........> "startVertexCollectionRestriction : 'frenchCity'}) RETURN [e.startVertex, e.vertex, " +

........> "e.distance]"

........> ).toArray();

show execution results

A route planner example, distance from Hamburg and Cologne to Lyon:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("FOR e IN GRAPH_DISTANCE_TO("

........> + "'routeplanner', [{_id: 'germanCity/Cologne'},{_id: 'germanCity/Hamburg'}], " +

........> "'frenchCity/Lyon', " +

........> "{weight : 'distance'}) RETURN [e.startVertex, e.vertex, e.distance]"

........> ).toArray();

show execution results

Graph measurements.

This section describes AQL functions to calculate various graph related measurements as defined in the mathematical graph theory.

GRAPH_ABSOLUTE_ECCENTRICITY

GRAPH_ABSOLUTE_ECCENTRICITY (graphName, vertexExample, options)

The GRAPH_ABSOLUTE_ECCENTRICITY function returns the eccentricity of the vertices defined by the examples.

The complexity of the function is described here.

Parameters

- graphName : The name of the graph as a string.

- vertexExample : An example for the desired vertices (see example).

- options (optional) : An object containing the following options:

- direction : The direction of the edges as a string. Possible values are outbound, inbound and any (default).

- edgeCollectionRestriction : One or multiple edge collection names. Only edges from these collections will be considered for the path.

- startVertexCollectionRestriction : One or multiple vertex collection names. Only vertices from these collections will be considered as start vertex of a path.

- endVertexCollectionRestriction : One or multiple vertex collection names. Only vertices from these collections will be considered as end vertex of a path.

- edgeExamples : A filter example for the edges in the shortest paths (see example).

- algorithm : The algorithm to calculate the shortest paths as a string. If vertex example is empty Floyd-Warshall is used as default, otherwise the default is Dijkstra

- weight : The name of the attribute of the edges containing the length as a string.

- defaultWeight : Only used with the option weight. If an edge does not have the attribute named as defined in option weight this default is used as length. If no default is supplied the default would be positive Infinity so the path and hence the eccentricity can not be calculated.

Examples

A route planner example, the absolute eccentricity of all locations.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_ABSOLUTE_ECCENTRICITY('routeplanner', {})").toArray();

show execution results

A route planner example, the absolute eccentricity of all locations. This considers the actual distances.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_ABSOLUTE_ECCENTRICITY("

........> +"'routeplanner', {}, {weight : 'distance'})").toArray();

show execution results

A route planner example, the absolute eccentricity of all German cities regarding only outbound paths.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_ABSOLUTE_ECCENTRICITY("

........> + "'routeplanner', {}, {startVertexCollectionRestriction : 'germanCity', " +

........> "direction : 'outbound', weight : 'distance'})").toArray();

show execution results

GRAPH_ECCENTRICITY

GRAPH_ECCENTRICITY (graphName, options)

The GRAPH_ECCENTRICITY function returns the normalized eccentricity of the graphs vertices

The complexity of the function is described here.

Parameters

- graphName : The name of the graph as a string.

- options (optional) : An object containing the following options:

- direction : The direction of the edges as a string. Possible values are outbound, inbound and any (default).

- algorithm : The algorithm to calculate the shortest paths as a string. Possible values are Floyd-Warshall (default) and Dijkstra.

- weight : The name of the attribute of the edges containing the length as a string.

- defaultWeight : Only used with the option weight. If an edge does not have the attribute named as defined in option weight this default is used as length. If no default is supplied the default would be positive Infinity so the path and hence the eccentricity can not be calculated.

Examples

A route planner example, the eccentricity of all locations.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_ECCENTRICITY('routeplanner')").toArray();

show execution results

A route planner example, the eccentricity of all locations. This considers the actual distances.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_ECCENTRICITY('routeplanner', {weight : 'distance'})"

........> ).toArray();

show execution results

GRAPH_ABSOLUTE_CLOSENESS

GRAPH_ABSOLUTE_CLOSENESS (graphName, vertexExample, options)

The GRAPH_ABSOLUTE_CLOSENESS function returns the closeness of the vertices defined by the examples.

The complexity of the function is described here.

Parameters

- graphName : The name of the graph as a string.

- vertexExample : An example for the desired vertices (see example).

- options : An object containing the following options:

- direction : The direction of the edges. Possible values are outbound, inbound and any (default).

- edgeCollectionRestriction : One or multiple edge collection names. Only edges from these collections will be considered for the path.

- startVertexCollectionRestriction : One or multiple vertex collection names. Only vertices from these collections will be considered as start vertex of a path.

- endVertexCollectionRestriction : One or multiple vertex collection names. Only vertices from these collections will be considered as end vertex of a path.

- edgeExamples : A filter example for the edges in the shortest paths ( see example).

- algorithm : The algorithm to calculate the shortest paths. Possible values are Floyd-Warshall (default) and Dijkstra.

- weight : The name of the attribute of the edges containing the length.

- defaultWeight : Only used with the option weight. If an edge does not have the attribute named as defined in option weight this default is used as length. If no default is supplied the default would be positive Infinity so the path and hence the eccentricity can not be calculated.

Examples

A route planner example, the absolute closeness of all locations.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_ABSOLUTE_CLOSENESS('routeplanner', {})").toArray();

show execution results

A route planner example, the absolute closeness of all locations. This considers the actual distances.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_ABSOLUTE_CLOSENESS("

........> +"'routeplanner', {}, {weight : 'distance'})").toArray();

show execution results

A route planner example, the absolute closeness of all German cities regarding only outbound paths.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_ABSOLUTE_CLOSENESS("

........> + "'routeplanner', {}, {startVertexCollectionRestriction : 'germanCity', " +

........> "direction : 'outbound', weight : 'distance'})").toArray();

show execution results

GRAPH_CLOSENESS

GRAPH_CLOSENESS (graphName, options)

The GRAPH_CLOSENESS function returns the normalized closeness of graphs vertices.

The complexity of the function is described here.

Parameters

- graphName : The name of the graph as a string.

- options : An object containing the following options:

- direction : The direction of the edges. Possible values are outbound, inbound and any (default).

- algorithm : The algorithm to calculate the shortest paths. Possible values are Floyd-Warshall (default) and Dijkstra.

- weight : The name of the attribute of the edges containing the length.

- defaultWeight : Only used with the option weight. If an edge does not have the attribute named as defined in option weight this default is used as length. If no default is supplied the default would be positive Infinity so the path and hence the eccentricity can not be calculated.

Examples

A route planner example, the closeness of all locations.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_CLOSENESS('routeplanner')").toArray();

show execution results

A route planner example, the closeness of all locations. This considers the actual distances.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_CLOSENESS("

........> +"'routeplanner', {weight : 'distance'})").toArray();

show execution results

A route planner example, the absolute closeness of all cities regarding only outbound paths.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_CLOSENESS("

........> + "'routeplanner',{direction : 'outbound', weight : 'distance'})"

........> ).toArray();

show execution results

GRAPH_ABSOLUTE_BETWEENNESS

GRAPH_ABSOLUTE_BETWEENNESS (graphName, options)

The GRAPH_ABSOLUTE_BETWEENNESS function returns the betweenness of all vertices in the graph.

The complexity of the function is described here.

- graphName : The name of the graph as a string.

- options : An object containing the following options:

- direction : The direction of the edges. Possible values are outbound, inbound and any (default).

- weight : The name of the attribute of the edges containing the length.

- defaultWeight : Only used with the option weight. If an edge does not have the attribute named as defined in option weight this default is used as length. If no default is supplied the default would be positive Infinity so the path and hence the betweenness can not be calculated.

Examples

A route planner example, the absolute betweenness of all locations.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_ABSOLUTE_BETWEENNESS('routeplanner', {})").toArray();

show execution results

A route planner example, the absolute betweenness of all locations. This considers the actual distances.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_ABSOLUTE_BETWEENNESS("

........> +"'routeplanner', {weight : 'distance'})").toArray();

show execution results

A route planner example, the absolute closeness regarding only outbound paths.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_ABSOLUTE_BETWEENNESS("

........> + "'routeplanner', {direction : 'outbound', weight : 'distance'})"

........> ).toArray();

show execution results

GRAPH_BETWEENNESS

GRAPH_BETWEENNESS (graphName, options)

The GRAPH_BETWEENNESS function returns the betweenness of graphs vertices.

The complexity of the function is described here.

Parameters

- graphName : The name of the graph as a string.

- options : An object containing the following options:

- direction : The direction of the edges. Possible values are outbound, inbound and any (default).

- weight : The name of the attribute of the edges containing the length.

- defaultWeight : Only used with the option weight. If an edge does not have the attribute named as defined in option weight this default is used as length. If no default is supplied the default would be positive Infinity so the path and hence the eccentricity can not be calculated.

Examples

A route planner example, the betweenness of all locations.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_BETWEENNESS('routeplanner')").toArray();

show execution results

A route planner example, the betweenness of all locations. This considers the actual distances.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_BETWEENNESS('routeplanner', {weight : 'distance'})").toArray();

show execution results

A route planner example, the betweenness regarding only outbound paths.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_BETWEENNESS("

........> + "'routeplanner', {direction : 'outbound', weight : 'distance'})").toArray();

show execution results

GRAPH_RADIUS

GRAPH_RADIUS (graphName, options)

The GRAPH_RADIUS function returns the radius of a graph.

The complexity of the function is described here.

- graphName : The name of the graph as a string.

- options : An object containing the following options:

- direction : The direction of the edges. Possible values are outbound, inbound and any (default).

- algorithm : The algorithm to calculate the shortest paths as a string. Possible values are Floyd-Warshall (default) and Dijkstra.

- weight : The name of the attribute of the edges containing the length.

- defaultWeight : Only used with the option weight. If an edge does not have the attribute named as defined in option weight this default is used as length. If no default is supplied the default would be positive Infinity so the path and hence the eccentricity can not be calculated.

Examples

A route planner example, the radius of the graph.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_RADIUS('routeplanner')").toArray();

[

1

]

A route planner example, the radius of the graph. This considers the actual distances.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_RADIUS('routeplanner', {weight : 'distance'})").toArray();

[

850

]

A route planner example, the radius of the graph regarding only outbound paths.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_RADIUS("

........> + "'routeplanner', {direction : 'outbound', weight : 'distance'})"

........> ).toArray();

[

550

]

GRAPH_DIAMETER

GRAPH_DIAMETER (graphName, options)

The GRAPH_DIAMETER function returns the diameter of a graph.

The complexity of the function is described here.

Parameters

- graphName : The name of the graph as a string.

- options : An object containing the following options:

- direction : The direction of the edges. Possible values are outbound, inbound and any (default).

- algorithm : The algorithm to calculate the shortest paths as a string. Possible values are Floyd-Warshall (default) and Dijkstra.

- weight : The name of the attribute of the edges containing the length.

- defaultWeight : Only used with the option weight. If an edge does not have the attribute named as defined in option weight this default is used as length. If no default is supplied the default would be positive Infinity so the path and hence the eccentricity can not be calculated.

Examples

A route planner example, the diameter of the graph.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_DIAMETER('routeplanner')").toArray();

[

1

]

A route planner example, the diameter of the graph. This considers the actual distances.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_DIAMETER('routeplanner', {weight : 'distance'})").toArray();

[

1200

]

A route planner example, the diameter of the graph regarding only outbound paths.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_DIAMETER("

........> + "'routeplanner', {direction : 'outbound', weight : 'distance'})"

........> ).toArray();

[

1200

]

Graph functions

AQL has the following functions to traverse loosely-structured graphs consisting of an edge collection and plus the vertices connected by it. If you have created a graph using the general-graph module you may want to use its more specific Graph operations instead.

Determining direct connections

Edges

EDGES(edgecollection, startvertex, direction, edgeexamples, options):

Return all edges connected to the vertex startvertex as an array. The possible values for direction are:

- outbound: Return all outbound edges

- inbound: Return all inbound edges

- any: Return outbound and inbound edges

The edgeexamples parameter can optionally be used to restrict the results to specific edge connections only. The matching is then done via the MATCHES function. To not restrict the result to specific connections, edgeexamples should be left unspecified.

Options is a JSON object that can contain the following properties:

- includeVertices: A boolean value, when set to true the return format will be modified. The function will then return an object with two attributes edge and vertex containing the respective elements.

In order to specify only options but no edgeExamples set edgeExamples to null.

Examples

EDGES(friendrelations, "friends/john", "outbound")

EDGES(friendrelations, "friends/john", "any", [ { "$label": "knows" } ])

EDGES(friendrelations, "friends/john", "any", [ { "$label": "knows" } ], {includeVertices: true})

EDGES(friendrelations, "friends/john", "any", null, {includeVertices: true})

Neighbors

NEIGHBORS(vertexcollection, edgecollection, startvertex, direction, edgeexamples, options):

Returns  _id  values of all distinct neighbors that are directly connected to the vertex startvertex as an array.

The possible values for direction are:

- outbound: Return all outbound edges

- inbound: Return all inbound edges

- any: Return outbound and inbound edges

The edgeexamples parameter can optionally be used to restrict the results to specific edge connections only. The matching is then done via the MATCHES function. To not restrict the result to specific connections, edgeexamples should be left unspecified.

The options parameter is an object and used to modify the result. Available options:

- includeData is a boolean value to define if the returned documents should be extracted instead of returning their ids only. The default is false.

Note: in ArangoDB versions prior to 2.6 NEIGHBORS returned the array of neighbor vertices together with their connecting edges including all attributes and not only the ids. To include the data and not only the ids set the includeData option to true in 2.6 and above. To create the result format of 2.5 and earlier please replace the NEIGHBORS function by EDGES with the option includeVertices set to true.

Examples

NEIGHBORS(friends, friendrelations, "friends/john", "outbound")

NEIGHBORS(users, usersrelations, "users/john", "any", [ { "$label": "recommends" } ] )

NEIGHBORS(users, usersrelations, "users/john", "any", [ { "$label": "recommends" } ], {includeData: true})

General Purpose Traversals

General purpose traversals with its extendability by visitor functions offer more possibilities over the newer AQL graph traversals, however unless you need some of these features you should prefer AQL graph traversals.

Traversal

TRAVERSAL(vertexcollection, edgecollection, startVertex, direction, options): Traverses the graph described by vertexcollection and edgecollection, starting at the vertex identified by id startVertex. Vertex connectivity is specified by the direction parameter:

- "outbound": Vertices are connected in _from to _to order

- "inbound": Vertices are connected in _to to _from order

- "any": Vertices are connected in both _to to _from and in _from to _to order

Additional options for the traversal can be provided via the options document:

- strategy: Defines the traversal strategy. Possible values are depthfirst and breadthfirst. Defaults to depthfirst

- order: Defines the traversal order: Possible values are preorder, postorder, and preorder-expander. Defaults to preorder

- itemOrder: Defines the level item order. Can be forward or backward. Defaults to forward

- minDepth: Minimum path depths for vertices to be included. This can be used to include only vertices in the result that are found after a certain minimum depth. Defaults to 0

- maxIterations: Maximum number of iterations in each traversal. This number can be set to prevent endless loops in traversal of cyclic graphs. When a traversal performs as many iterations as the maxIterations value, the traversal will abort with an error. If maxIterations is not set, a server-defined value will be used

- maxDepth: Maximum path depth for sub-edges expansion. This can be used to limit the depth of the traversal to a sensible amount. This should especially be used for big graphs to limit the traversal to some sensible amount, and for graphs containing cycles to prevent infinite traversals. The maximum depth defaults to 256, with the chance of this value being non-sensical. For several graphs, a much lower maximum depth is sensible, whereas for other, more array-oriented graphs a higher depth should be used

- paths: If true, the paths encountered during the traversal will also be returned along with each traversed vertex. If false, only the encountered vertices will be returned.

- uniqueness: An optional document with the following attributes:

- vertices:

- none: No vertex uniqueness is enforced

- global: A vertex may be visited at most once. This is the default.

- path: A vertex is visited only if not already contained in the current traversal path

- edges:

- none: No edge uniqueness is enforced

- global: An edge may be visited at most once. This is the default

- path: An edge is visited only if not already contained in the current traversal path

- followEdges: An optional array of example edge documents that the traversal will expand into. If no examples are given, the traversal will follow all edges. If one or many edge examples are given, the traversal will only follow an edge if it matches at least one of the specified examples. followEdges can also be a string with the name of an AQL user-defined function that should be responsible for checking if an edge should be followed. In this case, the AQL function will is expected to have the following signature:

function (config, vertex, edge, path)

The function is expected to return a boolean value. If it returns true, the edge will be followed. If false is returned, the edge will be ignored.

- filterVertices: An optional array of example vertex documents that the traversal will treat specially. If no examples are given, the traversal will handle all encountered vertices equally. If one or many vertex examples are given, the traversal will exclude any non-matching vertex from the result and/or not descend into it. Optionally, filterVertices can contain a string containing the name of a user-defined AQL function that should be responsible for filtering. If so, the AQL function is expected to have the following signature:

function (config, vertex, path)

If a custom AQL function is used for filterVertices, it is expected to return one of the following values:

- [ ]: Include the vertex in the result and descend into its connected edges

- [ "prune" ]: Will include the vertex in the result but not descend into its connected edges

- [ "exclude" ]: Will not include the vertex in the result but descend into its connected edges

- [ "prune", "exclude" ]: Will completely ignore the vertex and its connected edges

- vertexFilterMethod: Only useful in conjunction with filterVertices and if no user-defined AQL function is used. If specified, it will influence how vertices are handled that don't match the examples in filterVertices:

- [ "prune" ]: Will include non-matching vertices in the result but not descend into them

- [ "exclude" ]: Will not include non-matching vertices in the result but descend into them

- [ "prune", "exclude" ]: Will completely ignore the vertex and its connected edges

- visitor: If specified, must be a string containing the name of a custom AQL function that will be called whenever the traversal visits a vertex. The custom AQL function is expected to have the following signature:

function (config, result, vertex, path, connections)

There are two modes for custom AQL visitor functions, depending on the value of the option visitorReturnsResults. If it is set to true, then any value returned by the visitor will be appended to the list of results of the TRAVERSAL function. If visitorReturnsResults is set to false, then any value returned by the custom visitor function will be ignored. Instead, the visitor function is allowed to modify its result function parameter value in-place. The contents of this variable at the end of the traversal will then be the results of the TRAVERSAL function.

Note: The connections function parameter value will contain the edges connected to the vertex only if order was set to preorder-expander. Otherwise, the value of this parameter will be undefined.

The following custom visitor functions are predefined and can be used by specifying the function name in the visitor attribute:

- "_AQL::HASATTRIBUTESVISITOR": this visitor will produce an object if it contains the attributes specified in data.attributes for each visited vertex. If data.type is "all", then the object will only be returned if all specified attributes are present in the vertex, if data.type is "any", the object will be returned if the vertex contains any of the specified attributes. If the optional data.allowNull is set to  false , then the object will only be returned if an attribute is present and does not contain a  null  value. Otherwise an object will be returned already if the attribute is present.

- "_AQL::PROJECTINGVISITOR": this visitor will produce an object with the attributes specified in data.attributes for each visited vertex. This can be used to create a projection of each visited vertex' document.

- "_AQL::IDVISITOR": this visitor will return the _id attribute of each visited vertex.

- "_AQL::KEYVISITOR": this visitor will return the _key attribute of each visited vertex.

- "_AQL::COUNTINGVISITOR": this visitor will return a single number indicating the number of vertices visited.

- visitorReturnsResults: only useful in combination with a custom AQL visitor function. If set to true, the data returned by the visitor will be appended to the result. If set to false, any return value of the visitor function will be ignored. Instead, the visitor function can modify its result parameter value in-place. At the end of the traversal, result is expected to be an array.

- data: only useful in combination with custom AQL visitor or filter functions. This attribute can be used to pass arbitrary data into the custom visitor function. The value contained in the data attribute will be made available to the visitor and filterVertices functions in the config.data attribute.

By default, the result of the TRAVERSAL function is an array of traversed points. Each point is an object consisting of the following attributes:

- vertex: The vertex at the traversal point

- path: The path history for the traversal point. The path is a document with the attributes vertices and edges, which are both arrays. Note that path is only present in the result if the paths attribute is set in the options

When using a custom AQL function as a visitor, the result may have a different structure.

Examples

TRAVERSAL(friends, friendrelations, "friends/john", "outbound", {

strategy: "depthfirst",

order: "postorder",

itemOrder: "backward",

maxDepth: 6,

paths: true

})

// filtering on specific edges (by specifying example edges)

TRAVERSAL(friends, friendrelations, "friends/john", "outbound", {

strategy: "breadthfirst",

order: "preorder",

itemOrder: "forward",

followEdges: [ { type: "knows" }, { state: "FL" } ]

})

// filtering on specific edges and vertices

TRAVERSAL(friends, friendrelations, "friends/john", "outbound", {

strategy: "breadthfirst",

order: "preorder",

itemOrder: "forward",

followEdges: [ { type: "knows" }, { state: "FL" } ],

filterVertices: [ { isActive: true }, { isDeleted: false } ],

vertexFilterMethod: [ "prune", "exclude" ]

})

// using user-defined AQL functions for edge and vertex filtering

TRAVERSAL(friends, friendrelations, "friends/john", "outbound", {

followEdges: "myfunctions::checkedge",

filterVertices: "myfunctions::checkvertex"

})

// to register the custom AQL functions, execute something in the fashion of the

// following commands in arangosh once:

var aqlfunctions = require("org/arangodb/aql/functions");

// these are the actual filter functions

aqlfunctions.register("myfunctions::checkedge", function (config, vertex, edge, path) {

return (edge.type !== 'dislikes'); // don't follow these edges

}, false);

aqlfunctions.register("myfunctions::checkvertex", function (config, vertex, path) {

if (vertex.isDeleted || ! vertex.isActive) {

return [ "prune", "exclude" ]; // exclude these and don't follow them

}

return [ ]; // include everything else

}, false);

// using a custom visitor to return only leaf-nodes

TRAVERSAL(friends, friendrelations, "friends/john", "outbound", {

visitor: "myfunctions::leafnodevisitor",

visitorReturnsResults: true,

order: "preorder-expander"

})

// to register the custom AQL function visitor that only returns leaf nodes, run

// the following commands in arangosh once:

var aqlfunctions = require("org/arangodb/aql/functions");

aqlfunctions.register("myfunctions::leafnodevisitor", function (config, vertex, edge, path, connections) {

if (connected && connected.length === 0) {

// will put the vertex into the result if it does not have any further connections

return vertex;

}

}, false);

// using a custom visitor to only count leaf-nodes

TRAVERSAL(friends, friendrelations, "friends/john", "outbound", {

visitor: "myfunctions::countingleafnodevisitor",

visitorReturnsResults: false,

order: "preorder-expander"

})

// registering the custom counting leaf node visitor function

aqlfunctions.register("myfunctions::countingleafnodevisitor", function (config, vertex, edge, path, connections) {

if (result.length === 0) {

// first call, push 0 into the result

result.push(0);

}

if (connected && connected.length === 0) {

// found a leaf node. now increase our counter

result[0]++;

}

}, false);

// using one of the predefined visitor functions

TRAVERSAL(friends, friendrelations, "friends/john", "outbound", {

visitor: "_AQL::COUNTINGVISITOR"

})

// passing data into a visitor

TRAVERSAL(friends, friendrelations, "friends/john", "outbound", {

visitor: "_AQL::PROJECTINGVISITOR",

data: { attributes: [ "_id", "_key", "name" ] }

})

Traversal Tree

TRAVERSAL_TREE(vertexcollection, edgecollection, startVertex, direction, connectName, options): Traverses the graph described by vertexcollection and edgecollection, starting at the vertex identified by id startVertex and creates a hierarchical result. Vertex connectivity is establish by inserted an attribute which has the name specified via the connectName parameter. Connected vertices will be placed in this attribute as an array.

The options are the same as for the TRAVERSAL function, except that the result will be set up in a way that resembles a depth-first, pre-order visitation result. Thus, the strategy and order attributes of the options attribute will be ignored.

Examples

TRAVERSAL_TREE(friends, friendrelations, "friends/john", "outbound", "likes", {

itemOrder: "forward"

})

Shortest Path

SHORTEST_PATH(vertexcollection, edgecollection, startVertex, endVertex, direction, options): Determines the first shortest path from the startVertex to the endVertex. Both vertices must be present in the vertex collection specified in vertexcollection, and any connecting edges must be present in the collection specified by edgecollection. Vertex connectivity is specified by the direction parameter:

- "outbound": Vertices are connected in _from to _to order

- "inbound": Vertices are connected in _to to _from order

- "any": Vertices are connected in both _to to _from and in _from to _to order The search is aborted when a shortest path is found. Only the first shortest path will be returned. Any vertex will be visited at most once by the search.

Additional options for the shortest path can be provided via the options document:

- maxIterations: Maximum number of iterations in the search. This number can be set to bound long-running searches. When a search performs as many iterations as the maxIterations value, the search will abort with an error. If maxIterations is not set, a server-defined value may be used.

- includeData: Defines if the documents found on the path should be extracted or not. Default false, will only return the _id values.

- distance: a custom function with the following signature:

function (config, vertex1, vertex2, edge)

Both vertices and the connecting edge will be passed into the function. The function is expected to return a numeric value that expresses the distance between the two vertices. Higher values will mean higher distances, giving the connection a lower priority in further analysis. If this options is not specified all vertices are assumed to have the same distance (1) to each other. If a function name is specified, it must have been registered as a regular user-defined AQL function.

- weight: The name of a attribute weight attribute. If so this specific attribute will be examined on every edge on the path and will be used as a distance. In order to let this work properly you have to make sure to store only numerical values in this attribute. If the attribute is not present on any edge the value given in the defaultWeight option is used as a replacement. Using a weight attribute is only allowed in combination with defaultWeight.

- defaultWeight: Only useful in combination with an attribute name given in weight. If any edge does not have the attribute required for weight the numeric value defined by this option is used.

- followEdges: An optional array of example edge documents that the search will expand into. If no examples are given, the search will follow all edges. If one or many edge examples are given, the search will only follow an edge if it matches at least one of the specified examples. followEdges can also be a string with the name of an AQL user-defined function that should be responsible for checking if an edge should be followed. In this case, the AQL function will is expected to have the following signature:

function (config, vertex, edge, path)

The function is expected to return a boolean value. If it returns true, the edge will be followed. If false is returned, the edge will be ignored.

- filterVertices: An optional array of example vertex documents that the search will treat specially. If no examples are given, the search will handle all encountered vertices equally. If one or many vertex examples are given, the search will exclude the vertex from the result and/or not descend into it. Optionally, filterVertices can be a string containing the name of a user-defined AQL function that should be responsible for filtering. If so, the custom AQL function is expected to have the following signature:

function (config, vertex, path)

If a custom AQL function is used, it is expected to return one of the following values:

- [ ]: Include the vertex in the result and descend into its connected edges

- [ "prune" ]: Will include the vertex in the result but not descend into its connected edges

- [ "exclude" ]: Will not include the vertex in the result but descend into its connected edges

- [ "prune", "exclude" ]: Will completely ignore the vertex and its connected edges

- visitor: If specified, must be a string containing the name of a custom AQL function that will be called whenever the traversal visits a vertex. The custom AQL function is expected to have the following signature:

function (config, result, vertex, path, connections)

There are two modes for custom AQL visitor functions, depending on the value of the option visitorReturnsResults. If it is set to true, then any value returned by the visitor will be appended to the list of results of the TRAVERSAL function. If visitorReturnsResults is set to false, then any value returned by the custom visitor function will be ignored. Instead, the visitor function is allowed to modify its result function parameter value in-place. The contents of this variable at the end of the traversal will then be the results of the TRAVERSAL function.

Note: The connections function parameter value will contain the edges connected to the vertex only if order was set to preorder-expander. Otherwise, the value of this parameter will be undefined.

- visitorReturnsResults: only useful in combination with a custom AQL visitor function. If set to true, the data returned by the visitor will be appended to the result. If set to false, any return value of the visitor function will be ignored. Instead, the visitor function can modify its result parameter value in-place. At the end of the traversal, result is expected to be an array.

- data: only useful in combination with custom AQL visitor or filter functions. This attribute can be used to pass arbitrary data into the custom visitor function. The value contained in the data attribute will be made available to the visitor and filterVertices functions in the config.data attribute.

By default, the result of the SHORTEST_PATH function is an object consisting of the following attributes:

- vertices: The list of all vertices encountered on the path.

- edges: The list of all edges encountered on the path.

- distance: The total distance needed on this path. Using the given distance computation.

When using a custom AQL function as a visitor, the result may have a different structure.

Examples

SHORTEST_PATH(cities, motorways, "cities/CGN", "cities/MUC", "outbound")

// using a user-defined distance function

SHORTEST_PATH(cities, motorways, "cities/CGN", "cities/MUC", "outbound", {

paths: true,

distance: "myfunctions::citydistance"

})

// using a user-defined function to filter edges

SHORTEST_PATH(cities, motorways, "cities/CGN", "cities/MUC", "outbound", {

paths: true,

followEdges: "myfunctions::checkedge"

})

// to register a custom AQL distance function, execute something in the fashion of the

// following commands in arangosh once:

var aqlfunctions = require("org/arangodb/aql/functions");

// this is the actual distance function

aqlfunctions.register("myfunctions::distance", function (config, vertex1, vertex2, edge) {

return Math.sqrt(Math.pow(vertex1.x - vertex2.x) + Math.pow(vertex1.y - vertex2.y));

}, false);

// this is the filter function for the edges

aqlfunctions.register("myfunctions::checkedge", function (config, vertex, edge, path) {

return (edge.underConstruction === false); // don't follow these edges

}, false);

Other functions

Paths

PATHS(vertexcollection, edgecollection, direction, options): returns an array of paths through the graph defined by the nodes in the collection vertexcollection and edges in the collection edgecollection. For each vertex in vertexcollection, it will determine the paths through the graph depending on the value of direction:

- "outbound": Follow all paths that start at the current vertex and lead to another vertex

- "inbound": Follow all paths that lead from another vertex to the current vertex

- "any": Combination of "outbound" and "inbound" The default value for direction is "outbound". If specified, options must be a JavaScript object with the following optional attributes:

- minLength: specifies the minimum length of the paths to be returned. The default is 0.

- maxLength: specifies the maximum length of the paths to be returned. The default is 10.

- followcycles: if true, cyclic paths will be followed as well. This is turned off by default.

The result of the function is an array of paths. Paths of length 0 will also be returned. Each path is a document consisting of the following attributes:

- vertices: array of vertices visited along the path

- edges: array of edges visited along the path (may be empty)

- source: start vertex of path

- destination: destination vertex of path

Examples

PATHS(friends, friendrelations, "outbound")

FOR p IN PATHS(friends, friendrelations, "outbound", { minLength: 2, maxLength: 2 })

FILTER p.source._id == "users/123456"

RETURN p.vertices[*].name

Graph consistency

When using the graph management functions to remove vertices you have the guaranty that all referencing edges are also removed. However, if you use document features alone to remove vertices, no edge collections will be adjusted. This results in an edge with its  _from  or  _to  attribute referring to vanished vertices.

Now we query such a graph using the neighbors or the shortest path functions. If includeData wasn't enabled, the referred missing vertex will not be queried, and thus the result set will contain the referral to this missing vertex. If includeData is enabled, these missing vertices will be touched by the query.

In order to keep the result set consistent between includeData enabled or disabled, a  null  will be padded to fill the gap for each missing vertex.

Performance notes

When using any of AQL's general purpose traversal functions, please make sure that the graph does not contain cycles, or that you at least specify some maximum depth or uniqueness criteria for a traversal. In contrast AQL graph traversals won't trap on cycles.

If no bounds are set, a traversal may run into an endless loop in a cyclic graph or sub-graph. Even in a non-cyclic graph, traversing far into the graph may consume a lot of processing time and memory to create the result set.

By default, traversals are restricted to process a limited amount of vertices only and then abort. This is to protect the user from unintentionally running a traversal that will not abort or consume lots of resources. The maximum number of vertices that a traversal will process is specified by the optional maxIterations configuration value. If the number of vertices processed in a traversal reaches this cap will, the traversal will abort and throw a too many iterations exception.

Advanced features

Subqueries

Wherever an expression is allowed in AQL, a subquery can be placed. A subquery is a query part that can introduce its own local variables without affecting variables and values in its outer scope(s).

It is required that subqueries be put inside parentheses ( and ) to explicitly mark their start and end points:

FOR p IN persons

LET recommendations = (

FOR r IN recommendations

FILTER p.id == r.personId

SORT p.rank DESC

LIMIT 10

RETURN r

)

RETURN { person : p, recommendations : recommendations }

FOR p IN persons

COLLECT city = p.city INTO g

RETURN {

city : city,

numPersons : LENGTH(g),

maxRating: MAX(

FOR r IN g

RETURN r.p.rating

)}

Subqueries may also include other subqueries.

Array expansion

In order to access a named attribute from all elements in an array easily, AQL offers the shortcut operator [\*] for array variable expansion.

Using the [\*] operator with an array variable will iterate over all elements in the array, thus allowing to access a particular attribute of each element. It is required that the expanded variable is an array. The result of the [\*] operator is again an array.

To demonstrate the array expansion operator, let's go on with the following three example users documents:

[

{

name: "john",

age: 35,

friends: [

{ name: "tina", age: 43 },

{ name: "helga", age: 52 },

{ name: "alfred", age: 34 }

]

},

{

name: "yves",

age: 24,

friends: [

{ name: "sergei", age: 27 },

{ name: "tiffany", age: 25 }

]

},

{

name: "sandra",

age: 40,

friends: [

{ name: "bob", age: 32 },

{ name: "elena", age: 48 }

]

}

]

With the [\*] operator it becomes easy to query just the names of the friends for each user:

FOR u IN users

RETURN { name: u.name, friends: u.friends[*].name }

This will produce:

[

{ "name" : "john", "friends" : [ "tina", "helga", "alfred" ] },

{ "name" : "yves", "friends" : [ "sergei", "tiffany" ] },

{ "name" : "sandra", "friends" : [ "bob", "elena" ] }

]

This a shortcut for the longer, semantically equivalent query:

FOR u IN users

RETURN { name: u.name, friends: (FOR f IN u.friends RETURN f.name) }

While producing a result with the [\*] operator, it is also possible to filter while iterating over the array, and to create a projection using the current array element.

For example, to return only the names of friends that have an age value higher than the user herself an inline FILTER can be used:

FOR u IN users

RETURN { name: u.name, friends: u.friends[* FILTER CURRENT.age > u.age].name }

The pseudo-variable CURRENT can be used to access the current array element. The FILTER condition can refer to CURRENT or any variables valid in the outer scope.

To return a projection of the current element, use RETURN. If a FILTER is also present, RETURN must come later.

FOR u IN users

RETURN u.friends[* RETURN CONCAT(CURRENT.name, " is a friend of ", u.name)]

The above will return:

[

[

"tina is a friend of john",

"helga is a friend of john",

"alfred is a friend of john"

],

[

"sergei is a friend of yves",

"tiffany is a friend of yves"

],

[

"bob is a friend of sandra",

"elena is a friend of sandra"

]

]

Array contraction

In order to collapse (or flatten) results in nested arrays, AQL provides the [\*\*] operator. It works similar to the [\*] operator, but additionally collapses nested arrays. How many levels are collapsed is determined by the amount of \* characters used.

For example, the following query produces an array of friend names per user:

FOR u IN users

RETURN u.friends[*].name

As we have multiple users, the overall result is a nested array:

[

[

"tina",

"helga",

"alfred"

],

[

"sergei",

"tiffany"

],

[

"bob",

"elena"

]

]

If the goal is to get rid of the nested array, we can apply the [\*\*] operator on the result. Simplying appending [\*\*] to the query won't help, because *u.friends* is not a nested (multi-dimensional) array, but a simple (one-dimensional) array. Still, the [\*\*] can be used if it has access to a multi-dimensional nested result.

We can easily create a nested result like this:

RETURN (

FOR u IN users RETURN u.friends[*].name

)

By now appending the [\*\*] operator the end of the query, the query result becomes:

RETURN (

FOR u IN users RETURN u.friends[*].name

)[**]

[

[

"tina",

"helga",

"alfred",

"sergei",

"tiffany",

"bob",

"elena"

]

]

Extending AQL with User Functions

AQL comes with a built-in set of functions, but it is not a fully-featured programming language.

To add missing functionality or to simplify queries, users may add their own functions to AQL in the selected database. These functions can be written in JavaScript, and must be registered via the API; see Registering Functions.

In order to avoid conflicts with existing or future built-in function names, all user functions must be put into separate namespaces. Invoking a user functions is then possible by referring to the fully-qualified function name, which includes the namespace, too; see Conventions.

Conventions

Naming

The :: symbol is used inside AQL as the namespace separator. Using the namespace separator, users can create a multi-level hierarchy of function groups if required.

Examples:

RETURN myfunctions::myfunc()

RETURN myfunctions::math::random()

Note: As all function names in AQL, user function names are also case-insensitive.

Built-in AQL functions reside in the namespace _aql, which is also the default namespace to look in if an unqualified function name is found. Adding user functions to the _aql namespace is disallowed and will fail.

Variables and side effects

User functions can take any number of input arguments and should provide one result via a  return  statement. User functions should be kept purely functional and thus free of side effects and state, and state modification.

Modification of global variables is unsupported, as is changing the data of any collection from inside an AQL user function.

User function code is late-bound, and may thus not rely on any variables that existed at the time of declaration. If user function code requires access to any external data, it must take care to set up the data by itself.

All AQL user function-specific variables should be introduced with the  var  keyword in order to not accidentally access already defined variables from outer scopes. Not using the  var  keyword for own variables may cause side effects when executing the function.

Here is an example that may modify outer scope variables  i  and  name , making the function not side effects-free:

function (values) {

for (i = 0; i < values.length; ++i) {

name = values[i];

if (name === "foo") {

return i;

}

}

return null;

}

The above function can be made free of side effects by using the  var  or  let  keywords, so the variables become function-local variables:

function (values) {

for (var i = 0; i < values.length; ++i) {

var name = values[i];

if (name === "foo") {

return i;

}

}

return null;

}

Input parameters

In order to return a result, a user function should use a  return  instruction rather than modifying its input parameters.

AQL user functions are allowed to modify their input parameters for input parameters that are null, boolean, numeric or string values. Modifying these input parameter types inside a user function should be free of side effects. However, user functions should not modify input parameters if the parameters are arrays or objects and as such passed by reference, as that may modify variables and state outside of the user function itself.

Return values

User functions must only return primitive types (i.e. null, boolean values, numeric values, string values) or aggregate types (arrays or objects) composed of these types. Returning any other JavaScript object type from a user function may lead to undefined behavior and should be avoided.

Enforcing strict mode

By default, any user function code will not be executed in strict mode or strong mode. In order to make a user function being run in strict mode, use  use strict  explicitly inside the user function, e.g.:

function (values) {

"use strict"

for (var i = 0; i < values.length; ++i) {

var name = values[i];

if (name === "foo") {

return i;

}

}

return null;

}

Any violation of the strict mode will trigger a runtime error.

Miscellaneous

Internally, user functions are stored in a system collection named _aqlfunctions of the selected database. That means that by default they are excluded from dumps created with arangodump. To include AQL user functions in a dump, the dump should be started with the option --include-system-collections true.

Registering and Unregistering User Functions

AQL user functions can be registered in the selected database using the aqlfunctions object as follows:

var aqlfunctions = require("org/arangodb/aql/functions");

To register a function, the fully qualified function name plus the function code must be specified.

Register

register an AQL user function  aqlfunctions.register(name, code, isDeterministic)

Registers an AQL user function, identified by a fully qualified function name. The function code in code must be specified as a JavaScript function or a string representation of a JavaScript function. If the function code in code is passed as a string, it is required that the string evaluates to a JavaScript function definition.

If a function identified by name already exists, the previous function definition will be updated. Please also make sure that the function code does not violate the Conventions for AQL functions.

The isDeterministic attribute can be used to specify whether the function results are fully deterministic (i.e. depend solely on the input and are the same for repeated calls with the same input values). It is not used at the moment but may be used for optimizations later.

The registered function is stored in the selected database's system collection _aqlfunctions.

The function returns true when it updates/replaces an existing AQL function of the same name, and false otherwise. It will throw an exception when it detects syntactially invalid function code.

Examples

require("org/arangodb/aql/functions").register("myfunctions::temperature::celsiustofahrenheit",

function (celsius) {

return celsius * 1.8 + 32;

});

Unregister

delete an existing AQL user function  aqlfunctions.unregister(name)

Unregisters an existing AQL user function, identified by the fully qualified function name.

Trying to unregister a function that does not exist will result in an exception.

Examples

require("org/arangodb/aql/functions").unregister("myfunctions::temperature::celsiustofahrenheit");

Unregister Group

delete a group of AQL user functions  aqlfunctions.unregisterGroup(prefix)

Unregisters a group of AQL user function, identified by a common function group prefix.

This will return the number of functions unregistered.

Examples

require("org/arangodb/aql/functions").unregisterGroup("myfunctions::temperature");

require("org/arangodb/aql/functions").unregisterGroup("myfunctions");

To Array

list all AQL user functions  aqlfunctions.toArray()

Returns all previously registered AQL user functions, with their fully qualified names and function code.

The result may optionally be restricted to a specified group of functions by specifying a group prefix:

aqlfunctions.toArray(prefix)

Examples

To list all available user functions:

require("org/arangodb/aql/functions").toArray();

To list all available user functions in the myfunctions namespace:

require("org/arangodb/aql/functions").toArray("myfunctions");

To list all available user functions in the myfunctions::temperature namespace:

require("org/arangodb/aql/functions").toArray("myfunctions::temperature");

AQL Examples

This page contains some examples how to write queries in AQL. For better understandability the query results are also included directly below each query.

Simple queries

Following is a query that returns a string value. The result string is contained in an array because the result of every valid query is an array:

RETURN "this will be returned"

[

"this will be returned"

]

Here is a query that creates the cross products of two arrays and runs a projection on it, using a few of AQL's built-in functions:

FOR year in [ 2011, 2012, 2013 ]

FOR quarter IN [ 1, 2, 3, 4 ]

RETURN {

"y" : "year",

"q" : quarter,

"nice" : CONCAT(TO_STRING(quarter), "/", TO_STRING(year))

}

[

{ "y" : "year", "q" : 1, "nice" : "1/2011" },

{ "y" : "year", "q" : 2, "nice" : "2/2011" },

{ "y" : "year", "q" : 3, "nice" : "3/2011" },

{ "y" : "year", "q" : 4, "nice" : "4/2011" },

{ "y" : "year", "q" : 1, "nice" : "1/2012" },

{ "y" : "year", "q" : 2, "nice" : "2/2012" },

{ "y" : "year", "q" : 3, "nice" : "3/2012" },

{ "y" : "year", "q" : 4, "nice" : "4/2012" },

{ "y" : "year", "q" : 1, "nice" : "1/2013" },

{ "y" : "year", "q" : 2, "nice" : "2/2013" },

{ "y" : "year", "q" : 3, "nice" : "3/2013" },

{ "y" : "year", "q" : 4, "nice" : "4/2013" }

]

Collection-based queries

Normally you would want to run queries on data stored in collections. This section will provide several examples for that.

Example data

Some of the following example queries are executed on a collection users with the following initial data:

[

{ "id" : 100, "name" : "John", "age" : 37, "active" : true, "gender" : "m" },

{ "id" : 101, "name" : "Fred", "age" : 36, "active" : true, "gender" : "m" },

{ "id" : 102, "name" : "Jacob", "age" : 35, "active" : false, "gender" : "m" },

{ "id" : 103, "name" : "Ethan", "age" : 34, "active" : false, "gender" : "m" },

{ "id" : 104, "name" : "Michael", "age" : 33, "active" : true, "gender" : "m" },

{ "id" : 105, "name" : "Alexander", "age" : 32, "active" : true, "gender" : "m" },

{ "id" : 106, "name" : "Daniel", "age" : 31, "active" : true, "gender" : "m" },

{ "id" : 107, "name" : "Anthony", "age" : 30, "active" : true, "gender" : "m" },

{ "id" : 108, "name" : "Jim", "age" : 29, "active" : true, "gender" : "m" },

{ "id" : 109, "name" : "Diego", "age" : 28, "active" : true, "gender" : "m" },

{ "id" : 200, "name" : "Sophia", "age" : 37, "active" : true, "gender" : "f" },

{ "id" : 201, "name" : "Emma", "age" : 36,  "active" : true, "gender" : "f" },

{ "id" : 202, "name" : "Olivia", "age" : 35, "active" : false, "gender" : "f" },

{ "id" : 203, "name" : "Madison", "age" : 34, "active" : true, "gender": "f" },

{ "id" : 204, "name" : "Chloe", "age" : 33, "active" : true, "gender" : "f" },

{ "id" : 205, "name" : "Eva", "age" : 32, "active" : false, "gender" : "f" },

{ "id" : 206, "name" : "Abigail", "age" : 31, "active" : true, "gender" : "f" },

{ "id" : 207, "name" : "Isabella", "age" : 30, "active" : true, "gender" : "f" },

{ "id" : 208, "name" : "Mary", "age" : 29, "active" : true, "gender" : "f" },

{ "id" : 209, "name" : "Mariah", "age" : 28, "active" : true, "gender" : "f" }

]

For some of the examples, we'll also use a collection relations to store relationships between users. The example data for relations are as follows:

[

{ "from" : 209, "to" : 205, "type" : "friend" },

{ "from" : 206, "to" : 108, "type" : "friend" },

{ "from" : 202, "to" : 204, "type" : "friend" },

{ "from" : 200, "to" : 100, "type" : "friend" },

{ "from" : 205, "to" : 101, "type" : "friend" },

{ "from" : 209, "to" : 203, "type" : "friend" },

{ "from" : 200, "to" : 203, "type" : "friend" },

{ "from" : 100, "to" : 208, "type" : "friend" },

{ "from" : 101, "to" : 209, "type" : "friend" },

{ "from" : 206, "to" : 102, "type" : "friend" },

{ "from" : 104, "to" : 100, "type" : "friend" },

{ "from" : 104, "to" : 108, "type" : "friend" },

{ "from" : 108, "to" : 209, "type" : "friend" },

{ "from" : 206, "to" : 106, "type" : "friend" },

{ "from" : 204, "to" : 105, "type" : "friend" },

{ "from" : 208, "to" : 207, "type" : "friend" },

{ "from" : 102, "to" : 108, "type" : "friend" },

{ "from" : 207, "to" : 203, "type" : "friend" },

{ "from" : 203, "to" : 106, "type" : "friend" },

{ "from" : 202, "to" : 108, "type" : "friend" },

{ "from" : 201, "to" : 203, "type" : "friend" },

{ "from" : 105, "to" : 100, "type" : "friend" },

{ "from" : 100, "to" : 109, "type" : "friend" },

{ "from" : 207, "to" : 109, "type" : "friend" },

{ "from" : 103, "to" : 203, "type" : "friend" },

{ "from" : 208, "to" : 104, "type" : "friend" },

{ "from" : 105, "to" : 104, "type" : "friend" },

{ "from" : 103, "to" : 208, "type" : "friend" },

{ "from" : 203, "to" : 107, "type" : "boyfriend" },

{ "from" : 107, "to" : 203, "type" : "girlfriend" },

{ "from" : 208, "to" : 109, "type" : "boyfriend" },

{ "from" : 109, "to" : 208, "type" : "girlfriend" },

{ "from" : 106, "to" : 205, "type" : "girlfriend" },

{ "from" : 205, "to" : 106, "type" : "boyfriend" },

{ "from" : 103, "to" : 209, "type" : "girlfriend" },

{ "from" : 209, "to" : 103, "type" : "boyfriend" },

{ "from" : 201, "to" : 102, "type" : "boyfriend" },

{ "from" : 102, "to" : 201, "type" : "girlfriend" },

{ "from" : 206, "to" : 100, "type" : "boyfriend" },

{ "from" : 100, "to" : 206, "type" : "girlfriend" }

]

Things to consider when running queries on collections

Note that all documents created in the two collections will automatically get the following server-generated attributes:

- _id: A unique id, consisting of collection name and a server-side sequence value

- _key: The server sequence value

- _rev: The document's revision id

Whenever you run queries on the documents in the two collections, don't be surprised if these additional attributes are returned as well.

Please also note that with real-world data, you might want to create additional indexes on the data (left out here for brevity). Adding indexes on attributes that are used in FILTER statements may considerably speed up queries. Furthermore, instead of using attributes such as id, from and to, you might want to use the built-in _id, _from and _to attributes. Finally, edge collections provide a nice way of establishing references / links between documents. These features have been left out here for brevity as well.

Data-modification queries

The following operations can be used to modify data of multiple documents with one query. This is superior to fetching and updating the documents individually with multiple queries. However, if only a single document needs to be modified, ArangoDB's specialized data-modification operations for single documents might execute faster.

Updating documents

To update existing documents, we can either use the UPDATE or the REPLACE operation. UPDATE updates only the specified attributes in the found documents, and REPLACE completely replaces the found documents with the specified values.

We'll start with an UPDATE query that rewrites the gender attribute in all documents:

FOR u IN users

UPDATE u WITH { gender: TRANSLATE(u.gender, { m: 'male', f: 'female' }) } IN users

To add new attributes to existing documents, we can also use an UPDATE query. The following query adds an attribute numberOfLogins for all users with status active:

FOR u IN users

FILTER u.active == true

UPDATE u WITH { numberOfLogins: 0 } IN users

Existing attributes can also be updated based on their previous value:

FOR u IN users

FILTER u.active == true

UPDATE u WITH { numberOfLogins: u.numberOfLogins + 1 } IN users

The above query will only work if there was already a numberOfLogins attribute present in the document. If it is unsure whether there is a numberOfLogins attribute in the document, the increase must be made conditional:

FOR u IN users

FILTER u.active == true

UPDATE u WITH {

numberOfLogins: HAS(u, 'numberOfLogins') ? u.numberOfLogins + 1 : 1

} IN users

Updates of multiple attributes can be combined in a single query:

FOR u IN users

FILTER u.active == true

UPDATE u WITH {

lastLogin: DATE_NOW(),

numberOfLogins: HAS(u, 'numberOfLogins') ? u.numberOfLogins + 1 : 1

} IN users

Note than an update query might fail during execution, for example because a document to be updated does not exist. In this case, the query will abort at the first error. In single-server mode, all modifications done by the query will be rolled back as if they never happened.

Replacing documents

To not just partially update, but completely replace existing documents, use the REPLACE operation. The following query replaces all documents in the collection backup with the documents found in collection users. Documents common to both collections will be replaced. All other documents will remain unchanged. Documents are compared using their _key attributes:

FOR u IN users

REPLACE u IN backup

The above query will fail if there are documents in collection users that are not in collection backup yet. In this case, the query would attempt to replace documents that do not exist. If such case is detected while executing the query, the query will abort. In single-server mode, all changes made by the query will also be rolled back.

To make the query succeed for such case, use the ignoreErrors query option:

FOR u IN users

REPLACE u IN backup OPTIONS { ignoreErrors: true }

Removing documents

Deleting documents can be achieved with the REMOVE operation. To remove all users within a certain age range, we can use the following query:

FOR u IN users

FILTER u.active == true && u.age >= 35 && u.age <= 37

REMOVE u IN users

Creating documents

To create new documents, there is the INSERT operation. It can also be used to generate copies of existing documents from other collections, or to create synthetic documents (e.g. for testing purposes). The following query creates 1000 test users in collection users with some attributes set:

FOR i IN 1..1000

INSERT {

id: 100000 + i,

age: 18 + FLOOR(RAND() * 25),

name: CONCAT('test', TO_STRING(i)),

active: false,

gender: i % 2 == 0 ? 'male' : 'female'

} IN users

Copying data from one collection into another

To copy data from one collection into another, an INSERT operation can be used:

FOR u IN users

INSERT u IN backup

This will copy over all documents from collection users into collection backup. Note that both collections must already exist when the query is executed. The query might fail if backup already contains documents, as executing the insert might attempt to insert the same document (identified by _key attribute) again. This will trigger a unique key constraint violation and abort the query. In single-server mode, all changes made by the query will also be rolled back. To make such copy operation work in all cases, the target collection can be emptied before, using a REMOVE query.

Handling errors

In some cases it might be desirable to continue execution of a query even in the face of errors (e.g. "document not found"). To continue execution of a query in case of errors, there is the ignoreErrors option.

To use it, place an OPTIONS keyword directly after the data modification part of the query, e.g.

FOR u IN users

REPLACE u IN backup OPTIONS { ignoreErrors: true }

This will continue execution of the query even if errors occur during the REPLACE operation. It works similar for UPDATE, INSERT, and REMOVE.

Altering substructures

To modify lists in documents we have to work with temporary variables. We will collect the sublist in there and alter it. We choose a simple boolean filter condition to make the query better comprehensible.

First lets create a collection with a sample:

database = db._create('complexCollection')

database.save({

"topLevelAttribute" : "a",

"subList" : [

{

"attributeToAlter" : "oldValue",

"filterByMe" : true

},

{

"attributeToAlter" : "moreOldValues",

"filterByMe" : true

},

{

"attributeToAlter" : "unchangedValue",

"filterByMe" : false

}

]

})

Heres the Query which keeps the subList on alteredList to update it later:

FOR document in complexCollection

LET alteredList = (

FOR element IN document.subList

LET newItem = (! element.filterByMe ?

element :

MERGE(element, { attributeToAlter: "shiny New Value" }))

RETURN newItem)

UPDATE document WITH { subList:  alteredList } IN complexCollection

While the query as it is is now functional:

db.complexCollection.toArray()

[

{

"_id" : "complexCollection/392671569467",

"_key" : "392671569467",

"_rev" : "392799430203",

"topLevelAttribute" : "a",

"subList" : [

{

"filterByMe" : true,

"attributeToAlter" : "shiny New Value"

},

{

"filterByMe" : true,

"attributeToAlter" : "shiny New Value"

},

{

"filterByMe" : false,

"attributeToAlter" : "unchangedValue"

}

]

}

]

It will probably be soonish a performance bottleneck, since it modifies all documents in the collection regardless whether the values change or not. Therefore we want to only UPDATE the documents if we really change their value. Hence we employ a second FOR to test whether subList will be altered or not:

FOR document in complexCollection

LET willUpdateDocument = (

FOR element IN docToAlter.subList

FILTER element.filterByMe LIMIT 1 RETURN 1)

FILTER LENGTH(willUpdateDocument) > 0

LET alteredList = (

FOR element IN document.subList

LET newItem = (! element.filterByMe ?

element :

MERGE(element, { attributeToAlter: "shiny New Value" }))

RETURN newItem)

UPDATE document WITH { subList:  alteredList } IN complexCollection

Projections and Filters

Returning unaltered documents

To return three complete documents from collection users, the following query can be used:

FOR u IN users

LIMIT 0, 3

RETURN u

[

{

"_id" : "users/229886047207520",

"_rev" : "229886047207520",

"_key" : "229886047207520",

"active" : true,

"id" : 206,

"age" : 31,

"gender" : "f",

"name" : "Abigail"

},

{

"_id" : "users/229886045175904",

"_rev" : "229886045175904",

"_key" : "229886045175904",

"active" : true,

"id" : 101,

"age" : 36,

"name" : "Fred",

"gender" : "m"

},

{

"_id" : "users/229886047469664",

"_rev" : "229886047469664",

"_key" : "229886047469664",

"active" : true,

"id" : 208,

"age" : 29,

"name" : "Mary",

"gender" : "f"

}

]

Note that there is a LIMIT clause but no SORT clause. In this case it is not guaranteed which of the user documents are returned. Effectively the document return order is unspecified if no SORT clause is used, and you should not rely on the order in such queries.

Projections

To return a projection from the collection users use a modified RETURN instruction:

FOR u IN users

LIMIT 0, 3

RETURN {

"user" : {

"isActive" : u.active ? "yes" : "no",

"name" : u.name

}

}

[

{

"user" : {

"isActive" : "yes",

"name" : "John"

}

},

{

"user" : {

"isActive" : "yes",

"name" : "Anthony"

}

},

{

"user" : {

"isActive" : "yes",

"name" : "Fred"

}

}

]

Filters

To return a filtered projection from collection users, you can use the FILTER keyword. Additionally, a SORT clause is used to have the result returned in a specific order:

FOR u IN users

FILTER u.active == true && u.age >= 30

SORT u.age DESC

LIMIT 0, 5

RETURN {

"age" : u.age,

"name" : u.name

}

[

{

"age" : 37,

"name" : "Sophia"

},

{

"age" : 37,

"name" : "John"

},

{

"age" : 36,

"name" : "Emma"

},

{

"age" : 36,

"name" : "Fred"

},

{

"age" : 34,

"name" : "Madison"

}

]

Joins

So far we have only dealt with one collection (users) at a time. We also have a collection relations that stores relationships between users. We will now use this extra collection to create a result from two collections.

First of all, we'll query a few users together with their friends' ids. For that, we'll use all relations that have a value of friend in their type attribute. Relationships are established by using the friendOf and thisUser attributes in the relations collection, which point to the userId values in the users collection.

Join tuples

We'll start with a SQL-ish result set and return each tuple (user name, friends userId) separately. The AQL query to generate such result is:

FOR u IN users

FILTER u.active == true

LIMIT 0, 4

FOR f IN relations

FILTER f.type == "friend" && f.friendOf == u.userId

RETURN {

"user" : u.name,

"friendId" : f.thisUser

}

[

{

"user" : "Abigail",

"friendId" : 108

},

{

"user" : "Abigail",

"friendId" : 102

},

{

"user" : "Abigail",

"friendId" : 106

},

{

"user" : "Fred",

"friendId" : 209

},

{

"user" : "Mary",

"friendId" : 207

},

{

"user" : "Mary",

"friendId" : 104

},

{

"user" : "Mariah",

"friendId" : 203

},

{

"user" : "Mariah",

"friendId" : 205

}

]

We iterate over the collection users. Only the 'active' users will be examined. For each of these users we will search for up to 4 friends. We locate friends by comparing the userId of our current user with the friendOf attribute of the relations document. For each of those relations found we return the users name and the userId of the friend.

Horizontal lists

Note that in the above result, a user can be returned multiple times. This is the SQL way of returning data. If this is not desired, the friends' ids of each user can be returned in a horizontal list. This will return each user at most once.

The AQL query for doing so is:

FOR u IN users

FILTER u.active == true LIMIT 0, 4

RETURN {

"user" : u.name,

"friendIds" : (

FOR f IN relations

FILTER f.friendOf == u.userId && f.type == "friend"

RETURN f.thisUser

)

}

[

{

"user" : "Abigail",

"friendIds" : [

108,

102,

106

]

},

{

"user" : "Fred",

"friendIds" : [

209

]

},

{

"user" : "Mary",

"friendIds" : [

207,

104

]

},

{

"user" : "Mariah",

"friendIds" : [

203,

205

]

}

]

In this query we are still iterating over the users in the users collection and for each matching user we are executing a subquery to create the matching list of related users.

Self joins

To not only return friend ids but also the names of friends, we could "join" the users collection once more (something like a "self join"):

FOR u IN users

FILTER u.active == true

LIMIT 0, 4

RETURN {

"user" : u.name,

"friendIds" : (

FOR f IN relations

FILTER f.friendOf == u.userId && f.type == "friend"

FOR u2 IN users

FILTER f.thisUser == u2.useId

RETURN u2.name

)

}

[

{

"user" : "Abigail",

"friendIds" : [

"Jim",

"Jacob",

"Daniel"

]

},

{

"user" : "Fred",

"friendIds" : [

"Mariah"

]

},

{

"user" : "Mary",

"friendIds" : [

"Isabella",

"Michael"

]

},

{

"user" : "Mariah",

"friendIds" : [

"Madison",

"Eva"

]

}

]

This query will then again in term fetch the clear text name of the friend from the users collection. So here we iterate the users collection, and for each hit the relations collection, and for each hit once more the users collection.

Outer joins

Lets find the lonely people in our database - those without friends.

FOR user IN users

LET friendList = (

FOR f IN relations

FILTER f.friendOf == u.userId

RETURN 1

)

FILTER LENGTH(friendList) == 0

RETURN { "user" : user.name }

[

{

"user" : "Abigail"

},

{

"user" : "Fred"

}

]

So, for each user we pick the list of her friends and count them. The ones where count equals zero are the lonely people. Using RETURN 1 in the subquery saves even more precious CPU cycles and gives the optimizer more alternatives.

Pitfalls

Since we're free of schemata, there is by default no way to tell the format of the documents. So, if your documents don't contain an attribute, it defaults to null. We can however check our data for accuracy like this:

RETURN LENGTH(FOR u IN users FILTER u.userId == null RETURN 1)

[

10000

]

RETURN LENGTH(FOR f IN relations FILTER f.friendOf == null RETURN 1)

[

10000

]

So that the above queries return 10k matches each, the result of i.e. the Join tuples query will become 100.000.000 items large and will use much memory plus computation time. So it is generally a good idea to revalidate that the criteria for your join conditions exist.

Using indices on the properties can speed up the operation significantly. You can use the explain helper to revalidate your query actually uses them.

If you work with joins on edge collections you would typically aggregate over the internal fields _id, _from and _to (where _id equals userId, _from friendOf and _to would be thisUser in our examples). ArangoDB implicitly creates indices on them.

Grouping

To group results by arbitrary criteria, AQL provides the COLLECT keyword. COLLECT will perform a grouping, but no aggregation. Aggregation can still be added in the query if required.

Ensuring uniqueness

COLLECT can be used to make a result set unique. The following query will return each distinct  age  attribute value only once:

FOR u IN users

COLLECT age = u.age

RETURN age

This is grouping without tracking the group values, but just the group criterion (age) value.

Grouping can also be done on multiple levels using COLLECT:

FOR u IN users

COLLECT status = u.status, age = u.age

RETURN { status, age }

Alternatively RETURN DISTINCT can be used to make a result set unique. RETURN DISTINCT supports a single criterion only:

FOR u IN users

RETURN DISTINCT u.age

Note: the order of results is undefined for RETURN DISTINCT.

Fetching group values

To group users by age, and return the names of the users with the highest ages, we'll issue a query like this:

FOR u IN users

FILTER u.active == true

COLLECT age = u.age INTO usersByAge

SORT age DESC LIMIT 0, 5

RETURN {

age,

users : usersByAge[*].u.name

}

[

{

"age" : 37,

"users" : [

"John",

"Sophia"

]

},

{

"age" : 36,

"users" : [

"Fred",

"Emma"

]

},

{

"age" : 34,

"users" : [

"Madison"

]

},

{

"age" : 33,

"users" : [

"Chloe",

"Michael"

]

},

{

"age" : 32,

"users" : [

"Alexander"

]

}

]

The query will put all users together by their age attribute. There will be one result document per distinct age value (let aside the LIMIT). For each group, we have access to the matching document via the usersByAge variable introduced in the COLLECT statement.

Variable Expansion

The usersByAge variable contains the full documents found, and as we're only interested in user names, we'll use the expansion operator [\*] to extract just the name attribute of all user documents in each group.

The [\*] expansion operator is just a handy short-cut. Instead of usersByAge[\*].u.name we could also write:

FOR temp IN usersByAge

RETURN temp.u.name

Grouping by multiple criteria

To group by multiple criteria, we'll use multiple arguments in the COLLECT clause. For example, to group users by ageGroup (a derived value we need to calculate first) and then by gender, we'll do:

FOR u IN users

FILTER u.active == true

COLLECT ageGroup = FLOOR(u.age / 5) * 5,

gender = u.gender INTO group

SORT ageGroup DESC

RETURN {

ageGroup,

gender

}

[

{

"ageGroup" : 35,

"gender" : "f"

},

{

"ageGroup" : 35,

"gender" : "m"

},

{

"ageGroup" : 30,

"gender" : "f"

},

{

"ageGroup" : 30,

"gender" : "m"

},

{

"ageGroup" : 25,

"gender" : "f"

},

{

"ageGroup" : 25,

"gender" : "m"

}

]

Counting group values

If the goal is to count the number of values in each group, AQL provides the special COLLECT WITH COUNT INTO syntax. This is a simple variant for grouping with an additional group length calculation:

FOR u IN users

FILTER u.active == true

COLLECT ageGroup = FLOOR(u.age / 5) * 5,

gender = u.gender WITH COUNT INTO numUsers

SORT ageGroup DESC

RETURN {

ageGroup,

gender,

numUsers

}

[

{

"ageGroup" : 35,

"gender" : "f",

"numUsers" : 2

},

{

"ageGroup" : 35,

"gender" : "m",

"numUsers" : 2

},

{

"ageGroup" : 30,

"gender" : "f",

"numUsers" : 4

},

{

"ageGroup" : 30,

"gender" : "m",

"numUsers" : 4

},

{

"ageGroup" : 25,

"gender" : "f",

"numUsers" : 2

},

{

"ageGroup" : 25,

"gender" : "m",

"numUsers" : 2

}

]

Aggregation

Adding further aggregation is also simple in AQL by using an AGGREGATE clause in the COLLECT:

FOR u IN users

FILTER u.active == true

COLLECT ageGroup = FLOOR(u.age / 5) * 5,

gender = u.gender

AGGREGATE numUsers = LENGTH(1),

minAge = MIN(u.age),

maxAge = MAX(u.age)

SORT ageGroup DESC

RETURN {

ageGroup,

gender,

numUsers,

minAge,

maxAge

}

[

{

"ageGroup" : 35,

"gender" : "f",

"numUsers" : 2,

"minAge" : 36,

"maxAge" : 39,

},

{

"ageGroup" : 35,

"gender" : "m",

"numUsers" : 2,

"minAge" : 35,

"maxAge" : 39,

},

...

]

We have used the aggregate functions LENGTH here (it returns the length of a array). This is the equivalent to SQL's  SELECT g, COUNT(*) FROM ... GROUP BY g . In addition to LENGTH AQL also provides MAX, MIN, SUM and AVERAGE, VARIANCE_POPULATION, VARIANCE_SAMPLE, STDDEV_POPULATION and STDDEV_SAMPLE as basic aggregation functions.

In AQL all aggregation functions can be run on arrays only. If an aggregation function is run on anything that is not an array, a warning will be produced and the result will be null.

Using an AGGREGATE clause will ensure the aggregation is run while the groups are built in the collect operation. This is normally more efficient than collecting all group values for all groups and then doing a post-aggregation.

Post-aggregation

Aggregation can also be performed after a COLLECT operation using other AQL constructs, though performance-wise this is often inferior to using COLLECT with AGGREGATE.

The same query as before can be turned into a post-aggregation query as shown below. Note that this query will build and pass on all group values for all groups inside the variable g, and perform the aggregation at the latest possible stage:

FOR u IN users

FILTER u.active == true

COLLECT ageGroup = FLOOR(u.age / 5) * 5,

gender = u.gender INTO g

SORT ageGroup DESC

RETURN {

ageGroup,

gender,

numUsers : LENGTH(g[*]),

minAge : MIN(g[*].u.age),

maxAge : MAX(g[*].u.age)

}

[

{

"ageGroup" : 35,

"gender" : "f",

"numUsers" : 2,

"minAge" : 36,

"maxAge" : 39,

},

{

"ageGroup" : 35,

"gender" : "m",

"numUsers" : 2,

"minAge" : 35,

"maxAge" : 39,

},

...

]

This is in constrast to the previous query that used an AGGREGATE clause to perform the aggregation during the collect operation, at the earliest possible stage.

Post-filtering aggregated data

To filter the results of a grouping or aggregation operation (i.e. something similar to HAVING in SQL), simply add another FILTER clause after the COLLECT statement.

For example, to get the 3 ageGroups with the most users in them:

FOR u IN users

FILTER u.active == true

COLLECT ageGroup = FLOOR(u.age / 5) * 5 INTO group

LET numUsers = LENGTH(group)

FILTER numUsers > 2 /* group must contain at least 3 users in order to qualify */

SORT numUsers DESC

LIMIT 0, 3

RETURN {

"ageGroup" : ageGroup,

"numUsers" : numUsers,

"users" : group[*].u.name

}

[

{

"ageGroup" : 30,

"numUsers" : 8,

"users" : [

"Abigail",

"Madison",

"Anthony",

"Alexander",

"Isabella",

"Chloe",

"Daniel",

"Michael"

]

},

{

"ageGroup" : 25,

"numUsers" : 4,

"users" : [

"Mary",

"Mariah",

"Jim",

"Diego"

]

},

{

"ageGroup" : 35,

"numUsers" : 4,

"users" : [

"Fred",

"John",

"Emma",

"Sophia"

]

}

]

To increase readability, the repeated expression LENGTH(group) was put into a variable numUsers. The FILTER on numUsers is the equivalent an SQL HAVING clause.

Combining Graph Traversals

Finding the start vertex via a geo query

Our first example will locate the start vertex for a graph traversal via a geo index. We use the city graph and its geo indices:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> var bonn=[50.7340, 7.0998];

arangosh> db._query(`FOR startCity IN

........>             WITHIN(germanCity, @lat, @long, @radius)

........>               RETURN startCity`,

........>   {lat: bonn[0], long: bonn[1], radius: 400000}

........> ).toArray()

show execution results

We search all german cities in a range of 400 km around the ex-capital Bonn: Hamburg and Cologne. We won't find Paris since its in the  frenchCity  collection.

arangosh> db._query(`FOR startCity IN

........>             WITHIN(germanCity, @lat, @long, @radius)

........>               FOR v, e, p IN 1..1 OUTBOUND startCity

........>                 GRAPH 'routeplanner'

........>     RETURN {startcity: startCity._key, traversedCity: v}`,

........> {

........>  lat: bonn[0],

........>  long: bonn[1],

........>  radius: 400000

........> } ).toArray()

show execution results

The geo index query returns us  startCity  (Cologne and Hamburg) which we then use as starting point for our graph traversal. For simplicity we only return their direct neighbours. We format the return result so we can see from which  startCity  the traversal came.

Alternatively we could use a  LET  statement with a subquery to group the traversals by their  startCity  efficiently:

arangosh> db._query(`FOR startCity IN

........>            WITHIN(germanCity, @lat, @long, @radius)

........>              LET oneCity = (FOR v, e, p IN 1..1 OUTBOUND startCity

........>                GRAPH 'routeplanner' RETURN v)

........>              return {startCity: startCity._key, connectedCities: oneCity}`,

........> {

........>  lat: bonn[0],

........>  long: bonn[1],

........>  radius: 400000

........> } ).toArray();

show execution results

Finaly we clean up again:

arangosh> examples.dropGraph("routeplanner");

true

ArangoDB Graphs

First Steps with Graphs

A Graph consists of vertices and edges. Edges are stored as documents in edge collections. A vertex can be a document of a document collection or of an edge collection (so edges can be used as vertices). Which collections are used within a named graph is defined via edge definitions. A named graph can contain more than one edge definition, at least one is needed. Graphs allow you to structure your models in line with your domain and group them logically in collections and giving you the power to query them in the same graph queries.

Coming from a relational background - what is a graph?

In SQL you commonly have the construct of a relation table to store n:m relations between two data tables. An edge collection is somewhat similar to these relation tables; vertex collections resemble the data tables with the objects to connect. While simple graph queries with fixed number of hops via the relation table may be doable in SQL with several nested joins, graph databases can handle an arbitrary number of these hops over edge collections - this is called traversal. Also edges in one edge collection may point to several vertex collections. Its common to have attributes attached to edges, i.e. a label naming this interconnection. Edges have a direction, with their relations  _from  and  _to  pointing from one document to another document stored in vertex collections. In queries you can define in which directions the edge relations may be followed ( OUTBOUND :  _from  ->  _to ,  INBOUND :  _to  ->  _from ,  ANY :  _from  <->  _to ).

Named Graphs

Named graphs are completely managed by arangodb, and thus also visible in the webinterface. They use the ful spectrum of ArangoDBs graph features. You may access them via several interfaces.

- AQL Graph Operations with several flavors:

- AQL Graph traversals on both named and anonymous graphs

- AQL operations on named Graphs

- AQL operations on raw vertex & edge collections

- The javascript General Graph implementation, as you may use it in FOXX Services

- Graph Management; creating & manipualating graph definitions; inserting, updating and deleting vertices and edges into graphs

- Graph Functions for working with edges and vertices, to analyze them and their relations

- Fluent Query Interface; if you like to work with graphs via a fluent interface

- the RESTful General Graph interface used to implement graph management in client drivers

Manipulating collections of named graphs with regular document functions

The underlying collections of the named graphs are still accessible using the standard methods for collections. However the graph module adds an additional layer on top of these collections giving you the following guarantees:

- All modifications are executed transactional

- If you delete a vertex all edges will be deleted, you will never have loose ends

- If you insert an edge it is checked if the edge matches the edge definitions, your edge collections will only contain valid edges

These guarantees are lost if you access the collections in any other way than the graph module or AQL, so if you delete documents from your vertex collections directly, the edges pointing to them will be remain in place.

Anonymous graphs

Sometimes you may not need all the powers of named graphs, but some of its bits may be valuable to you. You may use AQL Graph Functions that are a little deeper explained in the traversals and the Working with Edges Chapter. Anonymous graphs don't have edge definitions describing which vertex collection is connected by which edge collection. The graph model has to be maintained in the client side code. This gives you more freedom than the strict named graphs.

When to choose anonymous or named graphs?

As noted above, named graphs ensure graph integrity, both when inserting or removing edges or vertices. So you won't encounter dangling edges, even if you use the same vertex collection in several named graphs. This involves more operations inside the database which don't come for free. Therefore anonymous graphs may be faster in many operations. So this question boils down to 'Can I afford the additional effort or do I need the warranty for integrity?'.

Backup and restore

For sure you want to have backups of your graph data, you can use ArangoDump to create the backup, and ArangoRestore to restore a backup into a new ArangoDB. You should however note that:

- you need the system collection  _graphs  if you backup named graphs.

- you need to backup the complete set of all edge and vertex collections your graph consists of. Partial dump/restore may not work.

Blueprint Graphs

Blueprint 2 graphs are a subset of named graphs. It only supports one edge and one vertex collection per graph. Both vertices and edges have to contain a label attribute; the Blueprints API defines a special  LABEL  attribute for this. We offer blueprints support via an api compatible java driver.

Example Graphs

ArangoDB comes with a set of easily graspable graphs that are used to demonstrate the APIs. You can use the  add samles  tab in the  create graph  window in the webinterface, or load the module  @arangodb/graph-examples/example-graph  in arangosh and use it to create instances of these graphs in your ArangoDB. Once you've created them, you can inspect them in the webinterface - which was used to create the pictures below.

The Knows_Graph

A set of persons knowing each other:

The knows graph consists of one vertex collection  persons  connected via one edge collection  knows . It will contain five persons Alice, Bob, Charlie, Dave and Eve. We will have the following directed relations:

- Alice knows Bob

- Bob knows Charlie

- Bob knows Dave

- Eve knows Alice

- Eve knows Bob

This is how we create it, inspect its vertices and edges, and drop it again:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("knows_graph");

arangosh> db.persons.toArray()

arangosh> db.knows.toArray();

arangosh> examples.dropGraph("knows_graph");

show execution results

The Social Graph

A set of persons and their relations:

This example has female and male persons as vertices in two vertex collections -  female  and  male . The edges are their connections in the  relation  edge collection. This is how we create it, inspect its vertices and edges, and drop it again:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> db.female.toArray()

arangosh> db.male.toArray()

arangosh> db.relation.toArray()

arangosh> examples.dropGraph("social");

show execution results

The City Graph

A set of european cities, and their fictional traveling distances as connections:

The example has the cities as vertices in several vertex collections -  germanCity  and  frenchCity . The edges are their interconnections in several edge collections  french / german / international Highway . This is how we create it, inspect its edges and vertices, and drop it again:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> db.frenchCity.toArray();

arangosh> db.germanCity.toArray();

arangosh> db.germanHighway.toArray();

arangosh> db.frenchHighway.toArray();

arangosh> db.internationalHighway.toArray();

arangosh> examples.dropGraph("routeplanner");

show execution results

The Traversal Graph

This graph was designed to demonstrate filters in traversals. It has some labels to filter on it.

The example has all its vertices in the circles collection, and an edges edge collection to connect them. Circles have uniq numeric labels. Edges have two boolean attributes (theFalse always being false, theTruth always being true) and a label sorting B - D to the left side, G - K to the right side. Left and right side split into Paths - at B and G which are each direct neighbours of the root-node A. Starting from A the graph has a depth of 3 on all its paths.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("traversalGraph");

arangosh> db.circles.toArray();

arangosh> db.edges.toArray();

arangosh> examples.dropGraph("traversalGraph");

show execution results

The World Graph

The world contry graph structures its nodes like that: world -> continet -> country -> capital. In some cases edge directions aren't forward (therefore it will be displayed disjunct in the graph viewer). It has two ways of creating it. One using the named graph utilities (worldCountry), one without (worldCountryUnManaged). It is used to demonstrate raw traversal operations.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("worldCountry");

arangosh> db.worldVertices.toArray();

arangosh> db.worldEdges.toArray();

arangosh> examples.dropGraph("worldCountry");

arangosh> var g = examples.loadGraph("worldCountryUnManaged");

arangosh> examples.dropGraph("worldCountryUnManaged");

show execution results

Cookbook examples

The above referenced chapters describe the various APIs of ArangoDBs graph engine with small examples. Our cookbook has some more real life examples:

- Traversing a graph in full depth

- Search for vertices of special type connecting a given subgraph

- Using an example vertex with the java driver

- Retrieving documents from ArangoDB without knowing the structure

- Using a custom visitor from node.js

- AQL Example Queries on an Actors and Movies Database

Higher volume graph examples

All of the above examples are rather small so they are easy cromprehensible and can demonstrate the way the functionality works. There are however several datasets freely available on the web that are a lot bigger. We collected some of them with import scripts so you may play around with them. Another huge graph is the Pokec social network from Slovakia that we used for performance testing on several databases; You will find importing scripts etc. in this blogpost.

Graphs

This chapter describes the general-graph module. It allows you to define a graph that is spread across several edge and document collections. This allows you to structure your models in line with your domain and group them logically in collections giving you the power to query them in the same graph queries. There is no need to include the referenced collections within the query, this module will handle it for you.

Three Steps to create a graph

- Create a graph

arangosh> var graph_module = require("@arangodb/general-graph");

arangosh> var graph = graph_module._create("myGraph");

arangosh> graph;

[ Graph myGraph EdgeDefinitions: [ ] VertexCollections: [ ] ]

- Add some vertex collections

arangosh> graph._addVertexCollection("shop");

arangosh> graph._addVertexCollection("customer");

arangosh> graph._addVertexCollection("pet");

arangosh> graph;

show execution results

- Define relations on the

arangosh> var rel = graph_module._relation("isCustomer", ["shop"], ["customer"]);

arangosh> graph._extendEdgeDefinitions(rel);

arangosh> graph;

[ Graph myGraph EdgeDefinitions: [

"isCustomer: [shop] -> [customer]"

] VertexCollections: [ ] ]

Graph Management

This chapter describes the javascript interface for creating and modifying named graphs. In order to create a non empty graph the functionality to create edge definitions has to be introduced first:

Edge Definitions

An edge definition is always a directed relation of a graph. Each graph can have arbitrary many relations defined within the edge definitions array.

Initialize the list

Create a list of edge definitions to construct a graph.

graph_module._edgeDefinitions(relation1, relation2, ..., relationN)

The list of edge definitions of a graph can be managed by the graph module itself. This function is the entry point for the management and will return the correct list.

Parameters

- relationX (optional): An object representing a definition of one relation in the graph

Examples

arangosh> var graph_module = require("org/arangodb/general-graph");

arangosh> directed_relation = graph_module._relation("lives_in", "user", "city");

arangosh> undirected_relation = graph_module._relation("knows", "user", "user");

arangosh> edgedefinitions = graph_module._edgeDefinitions(directed_relation, undirected_relation);

show execution results

Extend the list

Extend the list of edge definitions to construct a graph.

graph_module._extendEdgeDefinitions(edgeDefinitions, relation1, relation2, ..., relationN)

In order to add more edge definitions to the graph before creating this function can be used to add more definitions to the initial list.

Parameters

- edgeDefinitions (required): A list of relation definition objects.

- relationX (required): An object representing a definition of one relation in the graph

Examples

arangosh> var graph_module = require("org/arangodb/general-graph");

arangosh> directed_relation = graph_module._relation("lives_in", "user", "city");

arangosh> undirected_relation = graph_module._relation("knows", "user", "user");

arangosh> edgedefinitions = graph_module._edgeDefinitions(directed_relation);

arangosh> edgedefinitions = graph_module._extendEdgeDefinitions(undirected_relation);

show execution results

Relation

Define a directed relation.

graph_module._relation(relationName, fromVertexCollections, toVertexCollections)

The relationName defines the name of this relation and references to the underlying edge collection. The fromVertexCollections is an Array of document collections holding the start vertices. The toVertexCollections is an Array of document collections holding the target vertices. Relations are only allowed in the direction from any collection in fromVertexCollections to any collection in toVertexCollections.

Parameters

- relationName (required): The name of the edge collection where the edges should be stored. Will be created if it does not yet exist.

- fromVertexCollections (required): One or a list of collection names. Source vertices for the edges have to be stored in these collections. Collections will be created if they do not exist.

- toVertexCollections (required): One or a list of collection names. Target vertices for the edges have to be stored in these collections. Collections will be created if they do not exist.

Examples

arangosh> var graph_module = require("org/arangodb/general-graph");

arangosh> graph_module._relation("has_bought", ["Customer", "Company"], ["Groceries", "Electronics"]);

show execution results

arangosh> var graph_module = require("org/arangodb/general-graph");

arangosh> graph_module._relation("has_bought", "Customer", "Product");

show execution results

Undirected Relation

Warning: Deprecated

This function is deprecated and will be removed soon. Please use Relation instead.

Define an undirected relation.

graph_module._undirectedRelation(relationName, vertexCollections)

Defines an undirected relation with the name relationName using the list of vertexCollections. This relation allows the user to store edges in any direction between any pair of vertices within the vertexCollections.

Parameters

- relationName (required): The name of the edge collection where the edges should be stored. Will be created if it does not yet exist.

- vertexCollections (required): One or a list of collection names for which connections are allowed. Will be created if they do not exist.

Examples

To define simple relation with only one vertex collection:

arangosh> var graph_module = require("org/arangodb/general-graph");

arangosh> graph_module._undirectedRelation("friend", "user");

show execution results

To define a relation between several vertex collections:

arangosh> var graph_module = require("org/arangodb/general-graph");

arangosh> graph_module._undirectedRelation("marriage", ["female", "male"]);

show execution results

Directed Relation

Warning: Deprecated

This function is deprecated and will be removed soon. Please use Relation instead.

Define a directed relation.

graph_module._directedRelation(relationName, fromVertexCollections, toVertexCollections)

The relationName defines the name of this relation and references to the underlying edge collection. The fromVertexCollections is an Array of document collections holding the start vertices. The toVertexCollections is an Array of document collections holding the target vertices. Relations are only allowed in the direction from any collection in fromVertexCollections to any collection in toVertexCollections.

Parameters

- relationName (required): The name of the edge collection where the edges should be stored. Will be created if it does not yet exist.

- fromVertexCollections (required): One or a list of collection names. Source vertices for the edges have to be stored in these collections. Collections will be created if they do not exist.

- toVertexCollections (required): One or a list of collection names. Target vertices for the edges have to be stored in these collections. Collections will be created if they do not exist.

Examples

arangosh> var graph_module = require("org/arangodb/general-graph");

arangosh> graph_module._directedRelation("has_bought", ["Customer", "Company"], ["Groceries", "Electronics"]);

show execution results

Create a graph

After having introduced edge definitions a graph can be created.

Create a graph

graph_module._create(graphName, edgeDefinitions, orphanCollections)

The creation of a graph requires the name of the graph and a definition of its edges.

For every type of edge definition a convenience method exists that can be used to create a graph. Optionally a list of vertex collections can be added, which are not used in any edge definition. These collections are referred to as orphan collections within this chapter. All collections used within the creation process are created if they do not exist.

Parameters

- graphName (required): Unique identifier of the graph

- edgeDefinitions (optional): List of relation definition objects

- orphanCollections (optional): List of additional vertex collection names

Examples

Create an empty graph, edge definitions can be added at runtime:

arangosh> var graph_module = require("org/arangodb/general-graph");

arangosh> graph = graph_module._create("myGraph");

[ Graph myGraph EdgeDefinitions: [ ] VertexCollections: [ ] ]

Create a graph using an edge collection  edges  and a single vertex collection  vertices

arangosh> var graph_module = require("org/arangodb/general-graph");

arangosh> var edgeDefinitions = [ { collection: "edges", "from": [ "vertices" ], "to" : [ "vertices" ] } ];

arangosh> graph = graph_module._create("myGraph", edgeDefinitions);

[ Graph myGraph EdgeDefinitions: [

"edges: [vertices] -> [vertices]"

] VertexCollections: [ ] ]

Create a graph with edge definitions and orphan collections:

arangosh> var graph_module = require("org/arangodb/general-graph");

arangosh> graph = graph_module._create("myGraph",

........> [graph_module._relation("myRelation", ["male", "female"], ["male", "female"])], ["sessions"]);

show execution results

Complete Example to create a graph

Example Call:

arangosh> var graph_module = require("org/arangodb/general-graph");

arangosh> var edgeDefinitions = graph_module._edgeDefinitions();

arangosh> graph_module._extendEdgeDefinitions(edgeDefinitions, graph_module._relation("friend_of", "Customer", "Customer"));

arangosh> graph_module._extendEdgeDefinitions(

........> edgeDefinitions, graph_module._relation(

........> "has_bought", ["Customer", "Company"], ["Groceries", "Electronics"]));

arangosh> graph_module._create("myStore", edgeDefinitions);

[ Graph myStore EdgeDefinitions: [

"friend_of: [Customer] -> [Customer]",

"has_bought: [Company, Customer] -> [Electronics, Groceries]"

] VertexCollections: [ ] ]

alternative call:

arangosh> var graph_module = require("org/arangodb/general-graph");

arangosh>  var edgeDefinitions = graph_module._edgeDefinitions(

........>  graph_module._relation("friend_of", ["Customer"], ["Customer"]), graph_module._relation(

........> "has_bought", ["Customer", "Company"], ["Groceries", "Electronics"]));

arangosh> graph_module._create("myStore", edgeDefinitions);

[ Graph myStore EdgeDefinitions: [

"friend_of: [Customer] -> [Customer]",

"has_bought: [Company, Customer] -> [Electronics, Groceries]"

] VertexCollections: [ ] ]

List available graphs

List all graphs.

graph_module._list()

Lists all graph names stored in this database.

Examples

arangosh> var graph_module = require("org/arangodb/general-graph");

arangosh> graph_module._list();

[ ]

Load a graph

Get a graph

graph_module._graph(graphName)

A graph can be get by its name.

Parameters

- graphName (required): Unique identifier of the graph

Examples

Get a graph:

arangosh> var graph_module = require("org/arangodb/general-graph");

arangosh> graph = graph_module._graph("social");

[ Graph social EdgeDefinitions: [

"relation: [female, male] -> [female, male]"

] VertexCollections: [ ] ]

Remove a graph

Remove a graph

graph_module._drop(graphName, dropCollections)

A graph can be dropped by its name. This will automatically drop all collections contained in the graph as long as they are not used within other graphs. To drop the collections, the optional parameter drop-collections can be set to true.

Parameters

- graphName (required): Unique identifier of the graph

- dropCollections (optional): Define if collections should be dropped (default: false)

Examples

Drop a graph and keep collections:

arangosh> var graph_module = require("org/arangodb/general-graph");

arangosh> graph_module._drop("social");

true

arangosh> db._collection("female");

[ArangoCollection 1297084648, "female" (type document, status loaded)]

arangosh> db._collection("male");

[ArangoCollection 1297215720, "male" (type document, status loaded)]

arangosh> db._collection("relation");

[ArangoCollection 1297346792, "relation" (type edge, status loaded)]

arangosh> var graph_module = require("org/arangodb/general-graph");

arangosh> graph_module._drop("social", true);

true

arangosh> db._collection("female");

null

arangosh> db._collection("male");

null

arangosh> db._collection("relation");

null

Modify a graph definition during runtime

After you have created an graph its definition is not immutable. You can still add, delete or modify edge definitions and vertex collections.

Extend the edge definitions

Add another edge definition to the graph

graph._extendEdgeDefinitions(edgeDefinition)

Extends the edge definitions of a graph. If an orphan collection is used in this edge definition, it will be removed from the orphanage. If the edge collection of the edge definition to add is already used in the graph or used in a different graph with different from and/or to collections an error is thrown.

Parameters

- edgeDefinition (required): The relation definition to extend the graph

Examples

arangosh> var graph_module = require("org/arangodb/general-graph")

arangosh> var ed1 = graph_module._relation("myEC1", ["myVC1"], ["myVC2"]);

arangosh> var ed2 = graph_module._relation("myEC2", ["myVC1"], ["myVC3"]);

arangosh> var graph = graph_module._create("myGraph", [ed1]);

arangosh> graph._extendEdgeDefinitions(ed2);

Modify an edge definition

Modify an relation definition

graph_module._editEdgeDefinition(edgeDefinition)

Edits one relation definition of a graph. The edge definition used as argument will replace the existing edge definition of the graph which has the same collection. Vertex Collections of the replaced edge definition that are not used in the new definition will transform to an orphan. Orphans that are used in this new edge definition will be deleted from the list of orphans. Other graphs with the same edge definition will be modified, too.

Parameters

- edgeDefinition (required): The edge definition to replace the existing edge definition with the same attribute collection.

Examples

arangosh> var graph_module = require("org/arangodb/general-graph")

arangosh> var original = graph_module._relation("myEC1", ["myVC1"], ["myVC2"]);

arangosh> var modified = graph_module._relation("myEC1", ["myVC2"], ["myVC3"]);

arangosh> var graph = graph_module._create("myGraph", [original]);

arangosh> graph._editEdgeDefinitions(modified);

Delete an edge definition

Delete one relation definition

graph_module._deleteEdgeDefinition(edgeCollectionName, dropCollection)

Deletes a relation definition defined by the edge collection of a graph. If the collections defined in the edge definition (collection, from, to) are not used in another edge definition of the graph, they will be moved to the orphanage.

Parameters

- edgeCollectionName (required): Name of edge collection in the relation definition.

- dropCollection (optional): Define if the edge collection should be dropped. Default false.

Examples

Remove an edge definition but keep the edge collection:

arangosh> var graph_module = require("org/arangodb/general-graph")

arangosh> var ed1 = graph_module._relation("myEC1", ["myVC1"], ["myVC2"]);

arangosh> var ed2 = graph_module._relation("myEC2", ["myVC1"], ["myVC3"]);

arangosh> var graph = graph_module._create("myGraph", [ed1, ed2]);

arangosh> graph._deleteEdgeDefinition("myEC1");

arangosh> db._collection("myEC1");

[ArangoCollection 1884221672, "myEC1" (type edge, status loaded)]

Remove an edge definition and drop the edge collection:

arangosh> var graph_module = require("org/arangodb/general-graph")

arangosh> var ed1 = graph_module._relation("myEC1", ["myVC1"], ["myVC2"]);

arangosh> var ed2 = graph_module._relation("myEC2", ["myVC1"], ["myVC3"]);

arangosh> var graph = graph_module._create("myGraph", [ed1, ed2]);

arangosh> graph._deleteEdgeDefinition("myEC1", true);

arangosh> db._collection("myEC1");

null

Extend vertex Collections

Each graph can have an arbitrary amount of vertex collections, which are not part of any edge definition of the graph. These collections are called orphan collections. If the graph is extended with an edge definition using one of the orphans, it will be removed from the set of orphan collection automatically.

Add a vertex collection

Add a vertex collection to the graph

graph._addVertexCollection(vertexCollectionName, createCollection)

Adds a vertex collection to the set of orphan collections of the graph. If the collection does not exist, it will be created. If it is already used by any edge definition of the graph, an error will be thrown.

Parameters

- vertexCollectionName (required): Name of vertex collection.

- createCollection (optional): If true the collection will be created if it does not exist. Default: true.

Examples

arangosh> var graph_module = require("org/arangodb/general-graph");

arangosh> var ed1 = graph_module._relation("myEC1", ["myVC1"], ["myVC2"]);

arangosh> var graph = graph_module._create("myGraph", [ed1]);

arangosh> graph._addVertexCollection("myVC3", true);

Get the orphaned collections

Get all orphan collections

graph._orphanCollections()

Returns all vertex collections of the graph that are not used in any edge definition.

Examples

arangosh> var graph_module = require("org/arangodb/general-graph")

arangosh> var ed1 = graph_module._relation("myEC1", ["myVC1"], ["myVC2"]);

arangosh> var graph = graph_module._create("myGraph", [ed1]);

arangosh> graph._addVertexCollection("myVC3", true);

arangosh> graph._orphanCollections();

[

"myVC3"

]

Remove a vertex collection

Remove a vertex collection from the graph

graph._removeVertexCollection(vertexCollectionName, dropCollection)

Removes a vertex collection from the graph. Only collections not used in any relation definition can be removed. Optionally the collection can be deleted, if it is not used in any other graph.

Parameters

- vertexCollectionName (required): Name of vertex collection.

- dropCollection (optional): If true the collection will be dropped if it is not used in any other graph. Default: false.

Examples

arangosh> var graph_module = require("org/arangodb/general-graph")

arangosh> var ed1 = graph_module._relation("myEC1", ["myVC1"], ["myVC2"]);

arangosh> var graph = graph_module._create("myGraph", [ed1]);

arangosh> graph._addVertexCollection("myVC3", true);

arangosh> graph._addVertexCollection("myVC4", true);

arangosh> graph._orphanCollections();

arangosh> graph._removeVertexCollection("myVC3");

arangosh> graph._orphanCollections();

show execution results

Maniuplating Vertices

Save a vertex

Create a new vertex in vertexCollectionName

graph.vertexCollectionName.save(data)

Parameters

- data (required): JSON data of vertex.

Examples

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> graph.male.save({name: "Floyd", _key: "floyd"});

show execution results

Replace a vertex

Replaces the data of a vertex in collection vertexCollectionName

graph.vertexCollectionName.replace(vertexId, data, options)

Parameters

- vertexId (required): _id attribute of the vertex

- data (required): JSON data of vertex.

- options (optional): See collection documentation

Examples

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> graph.male.save({neym: "Jon", _key: "john"});

arangosh> graph.male.replace("male/john", {name: "John"});

show execution results

Update a vertex

Updates the data of a vertex in collection vertexCollectionName

graph.vertexCollectionName.update(vertexId, data, options)

Parameters

- vertexId (required): _id attribute of the vertex

- data (required): JSON data of vertex.

- options (optional): See collection documentation

Examples

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> graph.female.save({name: "Lynda", _key: "linda"});

arangosh> graph.female.update("female/linda", {name: "Linda", _key: "linda"});

show execution results

Remove a vertex

Removes a vertex in collection vertexCollectionName

graph.vertexCollectionName.remove(vertexId, options)

Additionally removes all ingoing and outgoing edges of the vertex recursively (see edge remove).

Parameters

- vertexId (required): _id attribute of the vertex

- options (optional): See collection documentation

Examples

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> graph.male.save({name: "Kermit", _key: "kermit"});

arangosh> db._exists("male/kermit")

arangosh> graph.male.remove("male/kermit")

arangosh> db._exists("male/kermit")

show execution results

Manipulating Edges

Save a new edge

Creates an edge from vertex from to vertex to in collection edgeCollectionName

graph.edgeCollectionName.save(from, to, data, options)

Parameters

- from (required): _id attribute of the source vertex

- to (required): _id attribute of the target vertex

- data (required): JSON data of the edge

- options (optional): See collection documentation

Examples

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> graph.relation.save("male/bob", "female/alice", {type: "married", _key: "bobAndAlice"});

show execution results

If the collections of from and to are not defined in an edge definition of the graph, the edge will not be stored.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> graph.relation.save(

........>  "relation/aliceAndBob",

........>   "female/alice",

........> {type: "married", _key: "bobAndAlice"});

[ArangoError 1906: invalid edge between relation/aliceAndBob and female/alice.]

Replace an edge

Replaces the data of an edge in collection edgeCollectionName

graph.edgeCollectionName.replace(edgeId, data, options)

Parameters

- edgeId (required): _id attribute of the edge

- data (required): JSON data of the edge

- options (optional): See collection documentation

Examples

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> graph.relation.save("female/alice", "female/diana", {typo: "nose", _key: "aliceAndDiana"});

arangosh> graph.relation.replace("relation/aliceAndDiana", {type: "knows"});

show execution results

Update an edge

Updates the data of an edge in collection edgeCollectionName

graph.edgeCollectionName.update(edgeId, data, options)

Parameters

- edgeId (required): _id attribute of the edge

- data (required): JSON data of the edge

- options (optional): See collection documentation

Examples

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> graph.relation.save("female/alice", "female/diana", {type: "knows", _key: "aliceAndDiana"});

arangosh> graph.relation.update("relation/aliceAndDiana", {type: "quarrelled", _key: "aliceAndDiana"});

show execution results

Remove an edge

Removes an edge in collection edgeCollectionName

graph.edgeCollectionName.remove(edgeId, options)

If this edge is used as a vertex by another edge, the other edge will be removed (recursively).

Parameters

- edgeId (required): _id attribute of the edge

- options (optional): See collection documentation

Examples

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> graph.relation.save("female/alice", "female/diana", {_key: "aliceAndDiana"});

arangosh> db._exists("relation/aliceAndDiana")

arangosh> graph.relation.remove("relation/aliceAndDiana")

arangosh> db._exists("relation/aliceAndDiana")

show execution results

Connect edges

Get all connecting edges between 2 groups of vertices defined by the examples

graph._connectingEdges(vertexExample, vertexExample2, options)

The function accepts an id, an example, a list of examples or even an empty example as parameter for vertexExample.

Parameters

- vertexExample1 (optional): See Definition of examples

- vertexExample2 (optional): See Definition of examples

- options (optional): An object defining further options. Can have the following values:

- edgeExamples: Filter the edges, see Definition of examples

- edgeCollectionRestriction : One or a list of edge-collection names that should be considered to be on the path.

- vertex1CollectionRestriction : One or a list of vertex-collection names that should be considered on the intermediate vertex steps.

- vertex2CollectionRestriction : One or a list of vertex-collection names that should be considered on the intermediate vertex steps.

Examples

A route planner example, all connecting edges between capitals.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._getConnectingEdges({isCapital : true}, {isCapital : true});

show execution results

Graph Functions

This chapter describes various functions on a graph. A lot of these accept a vertex (or edge) example as parameter as defined in the next section.

Examples will explain the API on the the city graph:

Definition of examples

For many of the following functions examples can be passed in as a parameter. Examples are used to filter the result set for objects that match the conditions. These examples can have the following values:

- null, there is no matching executed all found results are valid.

- A string, only results are returned, which _id equal the value of the string

- An example object, defining a set of attributes. Only results having these attributes are matched.

- A list containing example objects and/or strings. All results matching at least one of the elements in the list are returned.

Get vertices from edges.

Get vertex from of an edge

Get the source vertex of an edge

graph._fromVertex(edgeId)

Returns the vertex defined with the attribute _from of the edge with edgeId as its _id.

Parameters

- edgeId (required): _id attribute of the edge

Examples

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> graph._fromVertex("relation/aliceAndBob")

show execution results

Get vertex to of an edge

Get the target vertex of an edge

graph._toVertex(edgeId)

Returns the vertex defined with the attribute _to of the edge with edgeId as its _id.

Parameters

- edgeId (required): _id attribute of the edge

Examples

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> graph._toVertex("relation/aliceAndBob")

show execution results

_neighbors

Get all neighbors of the vertices defined by the example

graph._neighbors(vertexExample, options)

The function accepts an id, an example, a list of examples or even an empty example as parameter for vertexExample. The complexity of this method is O(n*m^x) with n being the vertices defined by the parameter vertexExamplex, m the average amount of neighbors and x the maximal depths. Hence the default call would have a complexity of O(n*m);

Parameters

- vertexExample (optional): See Definition of examples

- options (optional): An object defining further options. Can have the following values:

- direction: The direction of the edges. Possible values are outbound, inbound and any (default).

- edgeExamples: Filter the edges, see Definition of examples

- neighborExamples: Filter the neighbor vertices, see Definition of examples

- edgeCollectionRestriction : One or a list of edge-collection names that should be considered to be on the path.

- vertexCollectionRestriction : One or a list of vertex-collection names that should be considered on the intermediate vertex steps.

- minDepth: Defines the minimal number of intermediate steps to neighbors (default is 1).

- maxDepth: Defines the maximal number of intermediate steps to neighbors (default is 1).

Examples

A route planner example, all neighbors of capitals.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._neighbors({isCapital : true});

show execution results

A route planner example, all outbound neighbors of Hamburg.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._neighbors('germanCity/Hamburg', {direction : 'outbound', maxDepth : 2});

show execution results

_commonNeighbors

Get all common neighbors of the vertices defined by the examples.

graph._commonNeighbors(vertex1Example, vertex2Examples, optionsVertex1, optionsVertex2)

This function returns the intersection of graph_module._neighbors(vertex1Example, optionsVertex1) and graph_module._neighbors(vertex2Example, optionsVertex2). For parameter documentation see _neighbors.

The complexity of this method is O(n*m^x) with n being the maximal amount of vertices defined by the parameters vertexExamples, m the average amount of neighbors and x the maximal depths. Hence the default call would have a complexity of O(n*m);

Examples

A route planner example, all common neighbors of capitals.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._commonNeighbors({isCapital : true}, {isCapital : true});

show execution results

A route planner example, all common outbound neighbors of Hamburg with any other location which have a maximal depth of 2 :

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._commonNeighbors(

........>   'germanCity/Hamburg',

........>   {},

........>   {direction : 'outbound', maxDepth : 2},

........> {direction : 'outbound', maxDepth : 2});

show execution results

_countCommonNeighbors

Get the amount of common neighbors of the vertices defined by the examples.

graph._countCommonNeighbors(vertex1Example, vertex2Examples, optionsVertex1, optionsVertex2)

Similar to _commonNeighbors but returns count instead of the elements.

Examples

A route planner example, all common neighbors of capitals.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> var example = { isCapital: true };

arangosh> var options = { includeData: true };

arangosh> graph._countCommonNeighbors(example, example, options, options);

show execution results

A route planner example, all common outbound neighbors of Hamburg with any other location which have a maximal depth of 2 :

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> var options = { direction: 'outbound', maxDepth: 2, includeData: true };

arangosh> graph._countCommonNeighbors('germanCity/Hamburg', {}, options, options);

show execution results

_commonProperties

Get the vertices of the graph that share common properties.

graph._commonProperties(vertex1Example, vertex2Examples, options)

The function accepts an id, an example, a list of examples or even an empty example as parameter for vertex1Example and vertex2Example.

The complexity of this method is O(n) with n being the maximal amount of vertices defined by the parameters vertexExamples.

Parameters

- vertex1Examples (optional): Filter the set of source vertices, see Definition of examples

- vertex2Examples (optional): Filter the set of vertices compared to, see Definition of examples

- options (optional): An object defining further options. Can have the following values:

- vertex1CollectionRestriction : One or a list of vertex-collection names that should be searched for source vertices.

- vertex2CollectionRestriction : One or a list of vertex-collection names that should be searched for compare vertices.

- ignoreProperties : One or a list of attribute names of a document that should be ignored.

Examples

A route planner example, all locations with the same properties:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._commonProperties({}, {});

show execution results

A route planner example, all cities which share same properties except for population.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._commonProperties({}, {}, {ignoreProperties: 'population'});

show execution results

_countCommonProperties

Get the amount of vertices of the graph that share common properties.

graph._countCommonProperties(vertex1Example, vertex2Examples, options)

Similar to _commonProperties but returns count instead of the objects.

Examples

A route planner example, all locations with the same properties:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._countCommonProperties({}, {});

show execution results

A route planner example, all German cities which share same properties except for population.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._countCommonProperties({}, {}, {vertex1CollectionRestriction : 'germanCity',

........> vertex2CollectionRestriction : 'germanCity' ,ignoreProperties: 'population'});

show execution results

_paths

The _paths function returns all paths of a graph.

graph._paths(options)

This function determines all available paths in a graph.

The complexity of this method is O(n*n*m) with n being the amount of vertices in the graph and m the average amount of connected edges;

Parameters

- options (optional): An object containing options, see below:

- direction : The direction of the edges. Possible values are any, inbound and outbound (default).

- followCycles (optional) : If set to true the query follows cycles in the graph, default is false.

- minLength (optional) : Defines the minimal length a path must have to be returned (default is 0).

- maxLength (optional) : Defines the maximal length a path must have to be returned (default is 10).

Examples

Return all paths of the graph "social":

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("social");

arangosh> g._paths();

show execution results

Return all inbound paths of the graph "social" with a maximal length of 1 and a minimal length of 2:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("social");

arangosh> g._paths({direction : 'inbound', minLength : 1, maxLength :  2});

show execution results

_shortestPath

The _shortestPath function returns all shortest paths of a graph.

graph._shortestPath(startVertexExample, endVertexExample, options)

This function determines all shortest paths in a graph. The function accepts an id, an example, a list of examples or even an empty example as parameter for start and end vertex. If one wants to call this function to receive nearly all shortest paths for a graph the option algorithm should be set to Floyd-Warshall to increase performance. If no algorithm is provided in the options the function chooses the appropriate one (either Floyd-Warshall or Dijkstra) according to its parameters. The length of a path is by default the amount of edges from one start vertex to an end vertex. The option weight allows the user to define an edge attribute representing the length.

The complexity of the function is described here.

Parameters

- startVertexExample (optional): An example for the desired start Vertices (see Definition of examples).

- endVertexExample (optional): An example for the desired end Vertices (see Definition of examples).

- options (optional): An object containing options, see below:

- direction : The direction of the edges as a string. Possible values are outbound, inbound and any (default).

- edgeCollectionRestriction : One or multiple edge collection names. Only edges from these collections will be considered for the path.

- startVertexCollectionRestriction : One or multiple vertex collection names. Only vertices from these collections will be considered as start vertex of a path.

- endVertexCollectionRestriction : One or multiple vertex collection names. Only vertices from these collections will be considered as end vertex of a path.

- edgeExamples : A filter example for the edges in the shortest paths (see example).

- algorithm : The algorithm to calculate the shortest paths. If both start and end vertex examples are empty Floyd-Warshall is used, otherwise the default is Dijkstra

- weight : The name of the attribute of the edges containing the length as a string.

- defaultWeight : Only used with the option weight. If an edge does not have the attribute named as defined in option weight this default is used as length. If no default is supplied the default would be positive Infinity so the path could not be calculated.

Examples

A route planner example, shortest path from all german to all french cities:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> g._shortestPath({}, {}, {weight : 'distance', endVertexCollectionRestriction : 'frenchCity',

........> startVertexCollectionRestriction : 'germanCity'});

show execution results

A route planner example, shortest path from Hamburg and Cologne to Lyon:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> g._shortestPath([{_id: 'germanCity/Cologne'},{_id: 'germanCity/Munich'}], 'frenchCity/Lyon',

........> {weight : 'distance'});

show execution results

_distanceTo

The _distanceTo function returns all paths and there distance within a graph.

graph._distanceTo(startVertexExample, endVertexExample, options)

This function is a wrapper of graph._shortestPath. It does not return the actual path but only the distance between two vertices.

Examples

A route planner example, shortest distance from all german to all french cities:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> g._distanceTo({}, {}, {weight : 'distance', endVertexCollectionRestriction : 'frenchCity',

........> startVertexCollectionRestriction : 'germanCity'});

show execution results

A route planner example, shortest distance from Hamburg and Cologne to Lyon:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var g = examples.loadGraph("routeplanner");

arangosh> g._distanceTo([{_id: 'germanCity/Cologne'},{_id: 'germanCity/Munich'}], 'frenchCity/Lyon',

........> {weight : 'distance'});

show execution results

_absoluteEccentricity

Get the eccentricity of the vertices defined by the examples.

graph._absoluteEccentricity(vertexExample, options)

The function accepts an id, an example, a list of examples or even an empty example as parameter for vertexExample.

The complexity of the function is described here.

Parameters

- vertexExample (optional): Filter the vertices, see Definition of examples

- options (optional): An object defining further options. Can have the following values:

- direction: The direction of the edges. Possible values are outbound, inbound and any (default).

- edgeCollectionRestriction : One or a list of edge-collection names that should be considered to be on the path.

- startVertexCollectionRestriction : One or a list of vertex-collection names that should be considered for source vertices.

- endVertexCollectionRestriction : One or a list of vertex-collection names that should be considered for target vertices.

- edgeExamples: Filter the edges to be followed, see Definition of examples

- algorithm: The algorithm to calculate the shortest paths, possible values are Floyd-Warshall and Dijkstra.

- weight: The name of the attribute of the edges containing the weight.

- defaultWeight: Only used with the option weight. If an edge does not have the attribute named as defined in option weight this default is used as weight. If no default is supplied the default would be positive infinity so the path and hence the eccentricity can not be calculated.

Examples

A route planner example, the absolute eccentricity of all locations.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> db._query("RETURN GRAPH_ABSOLUTE_ECCENTRICITY("

........>   + "'routeplanner', {})"

........> ).toArray();

show execution results

A route planner example, the absolute eccentricity of all locations. This considers the actual distances.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._absoluteEccentricity({}, {weight : 'distance'});

show execution results

A route planner example, the absolute eccentricity of all cities regarding only outbound paths.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._absoluteEccentricity({}, {startVertexCollectionRestriction : 'germanCity',

........> direction : 'outbound', weight : 'distance'});

show execution results

_eccentricity

Get the normalized eccentricity of the vertices defined by the examples.

graph._eccentricity(vertexExample, options)

Similar to _absoluteEccentricity but returns a normalized result.

The complexity of the function is described here.

Examples

A route planner example, the eccentricity of all locations.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._eccentricity();

show execution results

A route planner example, the weighted eccentricity.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._eccentricity({weight : 'distance'});

show execution results

_absoluteCloseness

Get the closeness of the vertices defined by the examples.

graph._absoluteCloseness(vertexExample, options)

The function accepts an id, an example, a list of examples or even an empty example as parameter for vertexExample.

The complexity of the function is described here.

Parameters

- vertexExample (optional): Filter the vertices, see Definition of examples

- options (optional): An object defining further options. Can have the following values:

- direction: The direction of the edges. Possible values are outbound, inbound and any (default).

- edgeCollectionRestriction : One or a list of edge-collection names that should be considered to be on the path.

- startVertexCollectionRestriction : One or a list of vertex-collection names that should be considered for source vertices.

- endVertexCollectionRestriction : One or a list of vertex-collection names that should be considered for target vertices.

- edgeExamples: Filter the edges to be followed, see Definition of examples

- algorithm: The algorithm to calculate the shortest paths, possible values are Floyd-Warshall and Dijkstra.

- weight: The name of the attribute of the edges containing the weight.

- defaultWeight: Only used with the option weight. If an edge does not have the attribute named as defined in option weight this default is used as weight. If no default is supplied the default would be positive infinity so the path and hence the closeness can not be calculated.

Examples

A route planner example, the absolute closeness of all locations.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._absoluteCloseness({});

show execution results

A route planner example, the absolute closeness of all locations. This considers the actual distances.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._absoluteCloseness({}, {weight : 'distance'});

show execution results

A route planner example, the absolute closeness of all German Cities regarding only outbound paths.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._absoluteCloseness({}, {startVertexCollectionRestriction : 'germanCity',

........> direction : 'outbound', weight : 'distance'});

show execution results

_closeness

Get the normalized closeness of graphs vertices.

graph._closeness(options)

Similar to _absoluteCloseness but returns a normalized value.

The complexity of the function is described here.

Examples

A route planner example, the normalized closeness of all locations.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._closeness();

show execution results

A route planner example, the closeness of all locations. This considers the actual distances.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._closeness({weight : 'distance'});

show execution results

A route planner example, the closeness of all cities regarding only outbound paths.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._closeness({direction : 'outbound', weight : 'distance'});

show execution results

_absoluteBetweenness

Get the betweenness of all vertices in the graph.

graph._absoluteBetweenness(vertexExample, options)

The complexity of the function is described here.

Parameters

- vertexExample (optional): Filter the vertices, see Definition of examples

- options (optional): An object defining further options. Can have the following values:

- direction: The direction of the edges. Possible values are outbound, inbound and any (default).

- weight: The name of the attribute of the edges containing the weight.

- defaultWeight: Only used with the option weight. If an edge does not have the attribute named as defined in option weight this default is used as weight. If no default is supplied the default would be positive infinity so the path and hence the betweeness can not be calculated.

Examples

A route planner example, the absolute betweenness of all locations.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._absoluteBetweenness({});

show execution results

A route planner example, the absolute betweenness of all locations. This considers the actual distances.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._absoluteBetweenness({weight : 'distance'});

show execution results

A route planner example, the absolute betweenness of all cities regarding only outbound paths.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._absoluteBetweenness({direction : 'outbound', weight : 'distance'});

show execution results

_betweenness

Get the normalized betweenness of graphs vertices.

graph_module._betweenness(options)

Similar to _absoluteBetweeness but returns normalized values.

Examples

A route planner example, the betweenness of all locations.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._betweenness();

show execution results

A route planner example, the betweenness of all locations. This considers the actual distances.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._betweenness({weight : 'distance'});

show execution results

A route planner example, the betweenness of all cities regarding only outbound paths.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._betweenness({direction : 'outbound', weight : 'distance'});

show execution results

_radius

Get the radius of a graph.

graph._radius(options)

The complexity of the function is described here.

Parameters

- options (optional): An object defining further options. Can have the following values:

- direction: The direction of the edges. Possible values are outbound, inbound and any (default).

- algorithm: The algorithm to calculate the shortest paths, possible values are Floyd-Warshall and Dijkstra.

- weight: The name of the attribute of the edges containing the weight.

- defaultWeight: Only used with the option weight. If an edge does not have the attribute named as defined in option weight this default is used as weight. If no default is supplied the default would be positive infinity so the path and hence the radius can not be calculated.

Examples

A route planner example, the radius of the graph.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._radius();

1

A route planner example, the radius of the graph. This considers the actual distances.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._radius({weight : 'distance'});

850

A route planner example, the radius of the graph regarding only outbound paths.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._radius({direction : 'outbound', weight : 'distance'});

550

_diameter

Get the diameter of a graph.

graph._diameter(graphName, options)

The complexity of the function is described here.

Parameters

- options (optional): An object defining further options. Can have the following values:

- direction: The direction of the edges. Possible values are outbound, inbound and any (default).

- algorithm: The algorithm to calculate the shortest paths, possible values are Floyd-Warshall and Dijkstra.

- weight: The name of the attribute of the edges containing the weight.

- defaultWeight: Only used with the option weight. If an edge does not have the attribute named as defined in option weight this default is used as weight. If no default is supplied the default would be positive infinity so the path and hence the radius can not be calculated.

Examples

A route planner example, the diameter of the graph.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._diameter();

1

A route planner example, the diameter of the graph. This considers the actual distances.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._diameter({weight : 'distance'});

1200

A route planner example, the diameter of the graph regarding only outbound paths.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("routeplanner");

arangosh> graph._diameter({direction : 'outbound', weight : 'distance'});

1200

Fluent AQL Interface

This chapter describes a fluent interface to query your graph. The philosophy of this interface is to select a group of starting elements (vertices or edges) at first and from there on explore the graph with your query by selecting connected elements.

As an example you can start with a set of vertices, select their direct neighbors and finally their outgoing edges.

The result of this query will be the set of outgoing edges. For each part of the query it is possible to further refine the resulting set of elements by giving examples for them.

Examples will explain the API on the social graph:

Definition of examples

For many of the following functions examples can be passed in as a parameter. Examples are used to filter the result set for objects that match the conditions. These examples can have the following values:

- null, there is no matching executed all found results are valid.

- A string, only results are returned, which _id equal the value of the string

- An example object, defining a set of attributes. Only results having these attributes are matched.

- A list containing example objects and/or strings. All results matching at least one of the elements in the list are returned.

Starting Points

This section describes the entry points for the fluent interface. The philosophy of this module is to start with a specific subset of vertices or edges and from there on iterate over the graph.

Therefore you get exactly this two entry points:

- Select a set of edges

- Select a set of vertices

Edges

Select some edges from the graph.

graph._edges(examples)

Creates an AQL statement to select a subset of the edges stored in the graph. This is one of the entry points for the fluent AQL interface. It will return a mutable AQL statement which can be further refined, using the functions described below. The resulting set of edges can be filtered by defining one or more examples.

Parameters

- examples (optional): See Definition of examples

Examples

In the examples the toArray function is used to print the result. The description of this function can be found below.

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> graph._edges().toArray();

show execution results

To request filtered edges:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> graph._edges({type: "married"}).toArray();

show execution results

Vertices

Select some vertices from the graph.

graph._vertices(examples)

Creates an AQL statement to select a subset of the vertices stored in the graph. This is one of the entry points for the fluent AQL interface. It will return a mutable AQL statement which can be further refined, using the functions described below. The resulting set of edges can be filtered by defining one or more examples.

Parameters

- examples (optional): See Definition of examples

Examples

In the examples the toArray function is used to print the result. The description of this function can be found below.

To request unfiltered vertices:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> graph._vertices().toArray();

show execution results

To request filtered vertices:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> graph._vertices([{name: "Alice"}, {name: "Bob"}]).toArray();

show execution results

Working with the query cursor

The fluent query object handles cursor creation and maintenance for you. A cursor will be created as soon as you request the first result. If you are unhappy with the current result and want to refine it further you can execute a further step in the query which cleans up the cursor for you. In this interface you get the complete functionality available for general AQL cursors directly on your query. The cursor functionality is described in this section.

ToArray

Returns an array containing the complete result.

graph_query.toArray()

This function executes the generated query and returns the entire result as one array. ToArray does not return the generated query anymore and hence can only be the endpoint of a query. However keeping a reference to the query before executing allows to chain further statements to it.

Examples

To collect the entire result of a query toArray can be used:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices();

arangosh> query.toArray();

show execution results

HasNext

Checks if the query has further results.

graph_query.hasNext()

The generated statement maintains a cursor for you. If this cursor is already present hasNext() will use this cursors position to determine if there are further results available. If the query has not yet been executed hasNext() will execute it and create the cursor for you.

Examples

Start query execution with hasNext:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices();

arangosh> query.hasNext();

true

Iterate over the result as long as it has more elements:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices();

arangosh> while (query.hasNext()) {

........>   var entry = query.next();

........>   // Do something with the entry

........> }

Next

Request the next element in the result.

graph_query.next()

The generated statement maintains a cursor for you. If this cursor is already present next() will use this cursors position to deliver the next result. Also the cursor position will be moved by one. If the query has not yet been executed next() will execute it and create the cursor for you. It will throw an error of your query has no further results.

Examples

Request some elements with next:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices();

arangosh> query.next();

arangosh> query.next();

arangosh> query.next();

arangosh> query.next();

show execution results

The cursor is recreated if the query is changed:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices();

arangosh> query.next();

arangosh> query.edges();

arangosh> query.next();

show execution results

Count

Returns the number of returned elements if the query is executed.

graph_query.count()

This function determines the amount of elements to be expected within the result of the query. It can be used at the beginning of execution of the query before using next() or in between next() calls. The query object maintains a cursor of the query for you. count() does not change the cursor position.

Examples

To count the number of matched elements:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices();

arangosh> query.count();

4

Fluent queries

After the selection of the entry point you can now query your graph in a fluent way, meaning each of the functions on your query returns the query again. Hence it is possible to chain arbitrary many executions one after the other. In this section all available query statements are described.

Edges

Select all edges for the vertices selected before.

graph_query.edges(examples)

Creates an AQL statement to select all edges for each of the vertices selected in the step before. This will include inbound as well as outbound edges. The resulting set of edges can be filtered by defining one or more examples.

The complexity of this method is O(n*m^x) with n being the vertices defined by the parameter vertexExamplex, m the average amount of edges of a vertex and x the maximal depths. Hence the default call would have a complexity of O(n*m);

Parameters

- examples (optional): See Definition of examples

Examples

To request unfiltered edges:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices([{name: "Alice"}, {name: "Bob"}]);

arangosh> query.edges().toArray();

show execution results

To request filtered edges by a single example:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices([{name: "Alice"}, {name: "Bob"}]);

arangosh> query.edges({type: "married"}).toArray();

show execution results

To request filtered edges by multiple examples:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices([{name: "Alice"}, {name: "Bob"}]);

arangosh> query.edges([{type: "married"}, {type: "friend"}]).toArray();

show execution results

OutEdges

Select all outbound edges for the vertices selected before.

graph_query.outEdges(examples)

Creates an AQL statement to select all outbound edges for each of the vertices selected in the step before. The resulting set of edges can be filtered by defining one or more examples.

Parameters

- examples (optional): See Definition of examples

Examples

To request unfiltered outbound edges:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices([{name: "Alice"}, {name: "Bob"}]);

arangosh> query.outEdges().toArray();

show execution results

To request filtered outbound edges by a single example:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices([{name: "Alice"}, {name: "Bob"}]);

arangosh> query.outEdges({type: "married"}).toArray();

show execution results

To request filtered outbound edges by multiple examples:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices([{name: "Alice"}, {name: "Bob"}]);

arangosh> query.outEdges([{type: "married"}, {type: "friend"}]).toArray();

show execution results

InEdges

Select all inbound edges for the vertices selected before.

graph_query.inEdges(examples)

Creates an AQL statement to select all inbound edges for each of the vertices selected in the step before. The resulting set of edges can be filtered by defining one or more examples.

Parameters

- examples (optional): See Definition of examples

Examples

To request unfiltered inbound edges:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices([{name: "Alice"}, {name: "Bob"}]);

arangosh> query.inEdges().toArray();

show execution results

To request filtered inbound edges by a single example:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices([{name: "Alice"}, {name: "Bob"}]);

arangosh> query.inEdges({type: "married"}).toArray();

show execution results

To request filtered inbound edges by multiple examples:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices([{name: "Alice"}, {name: "Bob"}]);

arangosh> query.inEdges([{type: "married"}, {type: "friend"}]).toArray();

show execution results

Vertices

Select all vertices connected to the edges selected before.

graph_query.vertices(examples)

Creates an AQL statement to select all vertices for each of the edges selected in the step before. This includes all vertices contained in _from as well as _to attribute of the edges. The resulting set of vertices can be filtered by defining one or more examples.

Parameters

- examples (optional): See Definition of examples

Examples

To request unfiltered vertices:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._edges({type: "married"});

arangosh> query.vertices().toArray();

show execution results

To request filtered vertices by a single example:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._edges({type: "married"});

arangosh> query.vertices({name: "Alice"}).toArray();

show execution results

To request filtered vertices by multiple examples:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._edges({type: "married"});

arangosh> query.vertices([{name: "Alice"}, {name: "Charly"}]).toArray();

[ ]

FromVertices

Select all source vertices of the edges selected before.

graph_query.fromVertices(examples)

Creates an AQL statement to select the set of vertices where the edges selected in the step before start at. This includes all vertices contained in _from attribute of the edges. The resulting set of vertices can be filtered by defining one or more examples.

Parameters

- examples (optional): See Definition of examples

Examples

To request unfiltered source vertices:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._edges({type: "married"});

arangosh> query.fromVertices().toArray();

show execution results

To request filtered source vertices by a single example:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._edges({type: "married"});

arangosh> query.fromVertices({name: "Alice"}).toArray();

show execution results

To request filtered source vertices by multiple examples:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._edges({type: "married"});

arangosh> query.fromVertices([{name: "Alice"}, {name: "Charly"}]).toArray();

show execution results

ToVertices

Select all vertices targeted by the edges selected before.

graph_query.toVertices(examples)

Creates an AQL statement to select the set of vertices where the edges selected in the step before end in. This includes all vertices contained in _to attribute of the edges. The resulting set of vertices can be filtered by defining one or more examples.

Parameters

- examples (optional): See Definition of examples

Examples

To request unfiltered target vertices:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._edges({type: "married"});

arangosh> query.toVertices().toArray();

show execution results

To request filtered target vertices by a single example:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._edges({type: "married"});

arangosh> query.toVertices({name: "Bob"}).toArray();

show execution results

To request filtered target vertices by multiple examples:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._edges({type: "married"});

arangosh> query.toVertices([{name: "Bob"}, {name: "Diana"}]).toArray();

show execution results

Neighbors

Select all neighbors of the vertices selected in the step before.

graph_query.neighbors(examples, options)

Creates an AQL statement to select all neighbors for each of the vertices selected in the step before. The resulting set of vertices can be filtered by defining one or more examples.

Parameters

- examples (optional): See Definition of examples

- options (optional): An object defining further options. Can have the following values:

- direction: The direction of the edges. Possible values are outbound, inbound and any (default).

- edgeExamples: Filter the edges to be followed, see Definition of examples

- edgeCollectionRestriction : One or a list of edge-collection names that should be considered to be on the path.

- vertexCollectionRestriction : One or a list of vertex-collection names that should be considered on the intermediate vertex steps.

- minDepth: Defines the minimal number of intermediate steps to neighbors (default is 1).

- maxDepth: Defines the maximal number of intermediate steps to neighbors (default is 1).

Examples

To request unfiltered neighbors:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices({name: "Alice"});

arangosh> query.neighbors().toArray();

show execution results

To request filtered neighbors by a single example:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices({name: "Alice"});

arangosh> query.neighbors({name: "Bob"}).toArray();

show execution results

To request filtered neighbors by multiple examples:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._edges({type: "married"});

arangosh> query.vertices([{name: "Bob"}, {name: "Charly"}]).toArray();

[ ]

Restrict

Restricts the last statement in the chain to return only elements of a specified set of collections

graph_query.restrict(restrictions)

By default all collections in the graph are searched for matching elements whenever vertices and edges are requested. Using restrict after such a statement allows to restrict the search to a specific set of collections within the graph. Restriction is only applied to this one part of the query. It does not effect earlier or later statements.

Parameters

- restrictions (optional): Define either one or a list of collections in the graph. Only elements from these collections are taken into account for the result.

Examples

Request all directly connected vertices unrestricted:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices({name: "Alice"});

arangosh> query.edges().vertices().toArray();

show execution results

Apply a restriction to the directly connected vertices:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices({name: "Alice"});

arangosh> query.edges().vertices().restrict("female").toArray();

show execution results

Restriction of a query is only valid for collections known to the graph:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices({name: "Alice"});

arangosh> query.edges().vertices().restrict(["female", "male", "products"]).toArray();

[ArangoError 10: vertex collections: products are not known to the graph]

Filter

Filter the result of the query

graph_query.filter(examples)

This can be used to further specfiy the expected result of the query. The result set is reduced to the set of elements that matches the given examples.

Parameters

- examples (optional): See Definition of examples

Examples

Request vertices unfiltered:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._edges({type: "married"});

arangosh> query.toVertices().toArray();

show execution results

Request vertices filtered:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._edges({type: "married"});

arangosh> query.toVertices().filter({name: "Alice"}).toArray();

[ ]

Request edges unfiltered:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._edges({type: "married"});

arangosh> query.toVertices().outEdges().toArray();

show execution results

Request edges filtered:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._edges({type: "married"});

arangosh> query.toVertices().outEdges().filter({type: "married"}).toArray();

[ ]

Path

The result of the query is the path to all elements.

graph_query.path()

By defaut the result of the generated AQL query is the set of elements passing the last matches. So having a  vertices()  query as the last step the result will be set of vertices. Using  path()  as the last action before requesting the result will modify the result such that the path required to find the set vertices is returned.

Examples

Request the iteratively explored path using vertices and edges:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices({name: "Alice"});

arangosh> query.outEdges().toVertices().path().toArray();

show execution results

When requesting neighbors the path to these neighbors is expanded:

arangosh> var examples = require("org/arangodb/graph-examples/example-graph.js");

arangosh> var graph = examples.loadGraph("social");

arangosh> var query = graph._vertices({name: "Alice"});

arangosh> query.neighbors().path().toArray();

show execution results

Traversals

ArangoDB provides several ways to query graph data. Very simple operations can be composed with the low-level edge methods edges, inEdges, and outEdges for edge collections. These work on named and anonymous graphs. For more complex operations, ArangoDB provides predefined traversal objects.

Also Traversals have been added to AQL. Please read the chapter about AQL traversersals before you continue reading here. Most of the traversal cases are covered by AQL and will be executed in an optimized way. Only if the logic for your is too complex to be defined using AQL filters you can use the traversal object defined here which gives you complete programmatic access to the data.

For any of the following examples, we'll be using the example collections v and e, populated with continents, countries and capitals data listed below (see Example Data).

Starting from Scratch

ArangoDB provides the edges, inEdges, and outEdges methods for edge collections. These methods can be used to quickly determine if a vertex is connected to other vertices, and which. This functionality can be exploited to write very simple graph queries in JavaScript.

For example, to determine which edges are linked to the world vertex, we can use inEdges:

db.e.inEdges('v/world').forEach(function(edge) {

require("internal").print(edge._from, "->", edge.type, "->", edge._to);

});

inEdges will give us all ingoing edges for the specified vertex v/world. The result is a JavaScript array that we can iterate over and print the results:

v/continent-africa -> is-in -> v/world

v/continent-south-america -> is-in -> v/world

v/continent-asia -> is-in -> v/world

v/continent-australia -> is-in -> v/world

v/continent-europe -> is-in -> v/world

v/continent-north-america -> is-in -> v/world

Note: edges, inEdges, and outEdges return an array of edges. If we want to retrieve the linked vertices, we can use each edges' _from and _to attributes as follows:

db.e.inEdges('v/world').forEach(function(edge) {

require("internal").print(db._document(edge._from).name, "->", edge.type, "->", db._document(edge._to).name);

});

We are using the document method from the db object to retrieve the connected vertices now.

While this may be sufficient for one-level graph operations, writing a traversal by hand may become too complex for multi-level traversals.

Getting started

To use a traversal object, we first need to require the traversal module:

var traversal = require("org/arangodb/graph/traversal");

We then need to setup a configuration for the traversal and determine at which vertex to start the traversal:

var config = {

datasource: traversal.generalGraphDatasourceFactory("worldCountry"),

strategy: "depthfirst",

order: "preorder",

filter: traversal.visitAllFilter,

expander: traversal.inboundExpander,

maxDepth: 1

};

var startVertex = db._document("v/world");

Note: The startVertex needs to be a document, not only a document id.

We can then create a traverser and start the traversal by calling its traverse method. Note that traverse needs a result object, which it can modify in place:

var result = {

visited: {

vertices: [ ],

paths: [ ]

}

};

var traverser = new traversal.Traverser(config);

traverser.traverse(result, startVertex);

Finally, we can print the contents of the results object, limited to the visited vertices. We will only print the name and type of each visited vertex for brevity:

require("internal").print(result.visited.vertices.map(function(vertex) {

return vertex.name + " (" + vertex.type + ")";

}));

The full script, which includes all steps carried out so far is thus:

var traversal = require("org/arangodb/graph/traversal");

var config = {

datasource: traversal.generalGraphDatasourceFactory("worldCountry"),

strategy: "depthfirst",

order: "preorder",

filter: traversal.visitAllFilter,

expander: traversal.inboundExpander,

maxDepth: 1

};

var startVertex = db._document("v/world");

var result = {

visited: {

vertices: [ ],

paths: [ ]

}

};

var traverser = new traversal.Traverser(config);

traverser.traverse(result, startVertex);

require("internal").print(result.visited.vertices.map(function(vertex) {

return vertex.name + " (" + vertex.type + ")";

}));

The result is an array of vertices that were visited during the traversal, starting at the start vertex (i.e. v/world in our example):

[

"World (root)",

"Africa (continent)",

"Asia (continent)",

"Australia (continent)",

"Europe (continent)",

"North America (continent)",

"South America (continent)"

]

Note: The result is limited to vertices directly connected to the start vertex. We achieved this by setting the maxDepth attribute to 1. Not setting it would return the full array of vertices.

Traversal Direction

For the examples contained in this manual, we'll be starting the traversals at vertex v/world. Vertices in our graph are connected like this:

v/world <- is-in <- continent (Africa) <- is-in <- country (Algeria) <- is-in <- capital (Algiers)

To get any meaningful results, we must traverse the graph in inbound order. This means, we'll be following all incoming edges of to a vertex. In the traversal configuration, we have specified this via the expander attribute:

var config = {

...

expander: traversal.inboundExpander

};

For other graphs, we might want to traverse via the outgoing edges. For this, we can use the outboundExpander. There is also an anyExpander, which will follow both outgoing and incoming edges. This should be used with care and the traversal should always be limited to a maximum number of iterations (e.g. using the maxIterations attribute) in order to terminate at some point.

To invoke the default outbound expander for a graph, simply use the predefined function:

var config = {

...

expander: traversal.outboundExpander

};

Please note the outbound expander will not produce any output for the examples if we still start the traversal at the v/world vertex.

Still, we can use the outbound expander if we start somewhere else in the graph, e.g.

var traversal = require("org/arangodb/graph/traversal");

var config = {

datasource: traversal.generalGraphDatasourceFactory("world_graph"),

strategy: "depthfirst",

order: "preorder",

filter: traversal.visitAllFilter,

expander: traversal.outboundExpander

};

var startVertex = db._document("v/capital-algiers");

var result = {

visited: {

vertices: [ ],

paths: [ ]

}

};

var traverser = new traversal.Traverser(config);

traverser.traverse(result, startVertex);

require("internal").print(result.visited.vertices.map(function(vertex) {

return vertex.name + " (" + vertex.type + ")";

}));

The result is:

[

"Algiers (capital)",

"Algeria (country)",

"Africa (continent)",

"World (root)"

]

which confirms that now we're going outbound.

Traversal Strategy

Depth-first traversals

The visitation order of vertices is determined by the strategy and order attributes set in the configuration. We chose depthfirst and preorder, meaning the traverser will visit each vertex before handling connected edges (pre-order), and descend into any connected edges before processing other vertices on the same level (depth-first).

Let's remove the maxDepth attribute now. We'll now be getting all vertices (directly and indirectly connected to the start vertex):

var config = {

datasource: traversal.generalGraphDatasourceFactory("world_graph"),

strategy: "depthfirst",

order: "preorder",

filter: traversal.visitAllFilter,

expander: traversal.inboundExpander

};

var result = {

visited: {

vertices: [ ],

paths: [ ]

}

};

var traverser = new traversal.Traverser(config);

traverser.traverse(result, startVertex);

require("internal").print(result.visited.vertices.map(function(vertex) {

return vertex.name + " (" + vertex.type + ")";

}));

The result will be a longer array, assembled in depth-first, pre-order order. For each continent found, the traverser will descend into linked countries, and then into the linked capital:

[

"World (root)",

"Africa (continent)",

"Algeria (country)",

"Algiers (capital)",

"Angola (country)",

"Luanda (capital)",

"Botswana (country)",

"Gaborone (capital)",

"Burkina Faso (country)",

"Ouagadougou (capital)",

...

]

Let's switch the order attribute from preorder to postorder. This will make the traverser visit vertices after all connected vertices were visited (i.e. most distant vertices will be emitted first):

[

"Algiers (capital)",

"Algeria (country)",

"Luanda (capital)",

"Angola (country)",

"Gaborone (capital)",

"Botswana (country)",

"Ouagadougou (capital)",

"Burkina Faso (country)",

"Bujumbura (capital)",

"Burundi (country)",

"Yaounde (capital)",

"Cameroon (country)",

"N'Djamena (capital)",

"Chad (country)",

"Yamoussoukro (capital)",

"Cote d'Ivoire (country)",

"Cairo (capital)",

"Egypt (country)",

"Asmara (capital)",

"Eritrea (country)",

"Africa (continent)",

...

]

Breadth-first traversals

If we go back to preorder, but change the strategy to breadth-first and re-run the traversal, we'll see that the return order changes, and items on the same level will be returned adjacently:

[

"World (root)",

"Africa (continent)",

"Asia (continent)",

"Australia (continent)",

"Europe (continent)",

"North America (continent)",

"South America (continent)",

"Burkina Faso (country)",

"Burundi (country)",

"Cameroon (country)",

"Chad (country)",

"Algeria (country)",

"Angola (country)",

...

]

Note: The order of items returned for the same level is undefined. This is because there is no natural order of edges for a vertex with multiple connected edges. To explicitly set the order for edges on the same level, you can specify an edge comparator function with the sort attribute:

var config = {

...

sort: function (l, r) { return l._key < r._key ? 1 : -1; }

...

};

The arguments l and r are edge documents. This will traverse edges of the same vertex in backward _key order:

[

"World (root)",

"South America (continent)",

"North America (continent)",

"Europe (continent)",

"Australia (continent)",

"Asia (continent)",

"Africa (continent)",

"Ecuador (country)",

"Colombia (country)",

"Chile (country)",

"Brazil (country)",

"Bolivia (country)",

"Argentina (country)",

...

]

Note: This attribute only works for the usual expanders traversal.inboundExpander, traversal.outboundExpander, traversal.anyExpander and their corresponding "WithLabels" variants. If you are using custom expanders you have to organize the sorting within the specified expander.

Writing Custom Visitors

So far we have used much of the traverser's default functions. The traverser is very configurable and many of the default functions can be overridden with custom functionality.

For example, we have been using the default visitor function (which is always used if the configuration does not contain the visitor attribute). The default visitor function is called for each vertex in a traversal, and will push it into the result. This is the reason why the result variable looked different after the traversal, and needed to be initialized before the traversal was started.

Note that the default visitor (named  trackingVisitor ) will add every visited vertex into the result, including the full paths from the start vertex. This is useful for learning and debugging purposes, but should be avoided in production because it might produce (and copy) huge amounts of data. Instead, only those data should be copied into the result that are actually necessary.

The traverser comes with the following predefined visitors:

- trackingVisitor: this is the default visitor. It will copy all data of each visited vertex plus the full path information into the result. This can be slow if the result set is huge or vertices contain a lot of data.

- countingVisitor: this is a very lightweight visitor: all it does is increase a counter in the result for each vertex visited. Vertex data and paths will not be copied into the result.

- doNothingVisitor: if no action shall be carried out when a vertex is visited, this visitor can be employed. It will not do anything and will thus be fast. It can be used for performance comparisons with other visitors.

We can also write our own visitor function if we want to. The general function signature for visitor functions is as follows:

var config = {

...

visitor: function (config, result, vertex, path, connected) { ... }

};

Note: the connected parameter value will only be set if the traversal order is set to preorder-expander. Otherwise, this parameter won't be set by the traverser.

Visitor functions are not expected to return any values. Instead, they can modify the result variable (e.g. by pushing the current vertex into it), or do anything else. For example, we can create a simple visitor function that only prints information about the current vertex as we traverse:

var config = {

datasource: traversal.generalGraphDatasourceFactory("world_graph"),

strategy: "depthfirst",

order: "preorder",

filter: traversal.visitAllFilter,

expander: traversal.inboundExpander,

visitor: function (config, result, vertex, path) {

require("internal").print("visiting vertex", vertex.name);

}

};

var traverser = new traversal.Traverser(config);

traverser.traverse(undefined, startVertex);

To write a visitor that increments a counter each time a vertex is visited, we could write the following custom visitor:

config.visitor = function (config, result, vertex, path, connected) {

if (! result) {

result = { };

}

if (! result.hasOwnProperty('count')) {

result.count = 0;

}

++result.count;

}

Note that such visitor is already predefined (it's the countingVisitor described above). It can be used as follows:

config.visitor = traversal.countingVisitor;

Another example of a visitor is one that collects the  _id  values of all vertices visited:

config.visitor = function (config, result, vertex, path, connected) {

if (! result) {

result = { };

}

if (! result.hasOwnProperty("visited")) {

result.visited = { vertices: [ ] };

}

result.visited.vertices.push(vertex._id);

}

When the traversal order is set to preorder-expander, the traverser will pass a fifth parameter value into the visitor function. This parameter contains the connected edges of the visited vertex as an array. This can be handy because in this case the visitor will get all information about the vertex and the connected edges together.

For example, the following visitor can be used to print only leaf nodes (that do not have any further connected edges):

config.visitor = function (config, result, vertex, path, connected) {

if (connected && connected.length === 0) {

require("internal").print("found a leaf-node: ", vertex);

}

}

Note that for this visitor to work, the traversal order attribute needs to be set to the value preorder-expander.

Filtering Vertices and Edges

Filtering Vertices

So far we have printed or returned all vertices that were visited during the traversal. This is not always required. If the result shall be restrict to just specific vertices, we can use a filter function for vertices. It can be defined by setting the filter attribute of a traversal configuration, e.g.:

var config = {

filter: function (config, vertex, path) {

if (vertex.type !== 'capital') {

return 'exclude';

}

}

}

The above filter function will exclude all vertices that do not have a type value of capital. The filter function will be called for each vertex found during the traversal. It will receive the traversal configuration, the current vertex, and the full path from the traversal start vertex to the current vertex. The path consists of an array of edges, and an array of vertices. We could also filter everything but capitals by checking the length of the path from the start vertex to the current vertex. Capitals will have a distance of 3 from the v/world start vertex (capital -> is-in -> country -> is-in -> continent -> is-in -> world):

var config = {

...

filter: function (config, vertex, path) {

if (path.edges.length < 3) {

return 'exclude';

}

}

}

Note: If a filter function returns nothing (or undefined), the current vertex will be included, and all connected edges will be followed. If a filter function returns exclude the current vertex will be excluded from the result, and all still all connected edges will be followed. If a filter function returns prune, the current vertex will be included, but no connected edges will be followed.

For example, the following filter function will not descend into connected edges of continents, limiting the depth of the traversal. Still, continent vertices will be included in the result:

var config = {

...

filter: function (config, vertex, path) {

if (vertex.type === 'continent') {

return 'prune';

}

}

}

It is also possible to combine exclude and prune by returning an array with both values:

return [ 'exclude', 'prune' ];

Filtering Edges

It is possible to exclude certain edges from the traversal. To filter on edges, a filter function can be defined via the expandFilter attribute. The expandFilter is a function which is called for each edge during a traversal.

It will receive the current edge (edge variable) and the vertex which the edge connects to (in the direction of the traversal). It also receives the current path from the start vertex up to the current vertex (excluding the current edge and the vertex the edge points to).

If the function returns true, the edge will be followed. If the function returns false, the edge will not be followed. Here is a very simple custom edge filter function implementation, which simply includes edges if the (edges) path length is less than 1, and will exclude any other edges. This will effectively terminate the traversal after the first level of edges:

var config = {

...

expandFilter: function (config, vertex, edge, path) {

return (path.edges.length < 1);

}

};

Writing Custom Expanders

The edges connected to a vertex are determined by the expander. So far we have used a default expander (the default inbound expander to be precise). The default inbound expander simply enumerates all connected ingoing edges for a vertex, based on the edge collection specified in the traversal configuration.

There is also a default outbound expander, which will enumerate all connected outgoing edges. Finally, there is an any expander, which will follow both ingoing and outgoing edges.

If connected edges must be determined in some different fashion for whatever reason, a custom expander can be written and registered by setting the expander attribute of the configuration. The expander function signature is as follows:

var config = {

...

expander: function (config, vertex, path) { ... }

}

It is the expander's responsibility to return all edges and vertices directly connected to the current vertex (which is passed via the vertex variable). The full path from the start vertex up to the current vertex is also supplied via the path variable. An expander is expected to return an array of objects, which need to have an edge and a vertex attribute each.

Note: If you want to rely on a particular order in which the edges are traversed, you have to sort the edges returned by your expander within the code of the expander. The functions to get outbound, inbound or any edges from a vertex do not guarantee any particular order!

A custom implementation of an inbound expander could look like this (this is a non-deterministic expander, which randomly decides whether or not to include connected edges):

var config = {

...

expander: function (config, vertex, path) {

var connected = [ ];

var datasource = config.datasource;

datasource.getInEdges(vertex._id).forEach(function (edge) {

if (Math.random() >= 0.5) {

connected.push({ edge: edge, vertex: (edge._from) });

}

});

return connected;

}

};

A custom expander can also be used as an edge filter because it has full control over which edges will be returned.

Following are two examples of custom expanders that pick edges based on attributes of the edges and the connected vertices.

Finding the connected edges / vertices based on an attribute when in the connected vertices. The goal is to follow the edge that leads to the vertex with the highest value in the when attribute:

var config = {

...

expander: function (config, vertex, path) {

var datasource = config.datasource;

// determine all outgoing edges

var outEdges = datasource.getOutEdges(vertex);

if (outEdges.length === 0) {

return [ ];

}

var data = [ ];

outEdges.forEach(function (edge) {

data.push({ edge: edge, vertex: datasource.getInVertex(edge) });

});

// sort outgoing vertices according to "when" attribute value

data.sort(function (l, r) {

if (l.vertex.when === r.vertex.when) {

return 0;

}

return (l.vertex.when < r.vertex.when ? 1 : -1);

});

// pick first vertex found (with highest "when" attribute value)

return [ data[0] ];

}

...

};

Finding the connected edges / vertices based on an attribute when in the edge itself. The goal is to pick the one edge (out of potentially many) that has the highest when attribute value:

var config = {

...

expander: function (config, vertex, path) {

var datasource = config.datasource;

// determine all outgoing edges

var outEdges = datasource.getOutEdges(vertex);

if (outEdges.length === 0) {

return [ ]; // return an empty array

}

// sort all outgoing edges according to "when" attribute

outEdges.sort(function (l, r) {

if (l.when === r.when) {

return 0;

}

return (l.when < r.when ? -1 : 1);

});

// return first edge (the one with highest "when" value)

var edge = outEdges[0];

try {

var v = datasource.getInVertex(edge);

return [ { edge: edge, vertex: v } ];

}

catch (e) { }

return [ ];

}

...

};

Handling Uniqueness

Graphs may contain cycles. To be on top of what happens when a traversal encounters a vertex or an edge it has already visited, there are configuration options.

The default configuration is to visit every vertex, regardless of whether it was already visited in the same traversal. However, edges will by default only be followed if they are not already present in the current path.

Imagine the following graph which contains a cycle:

A -> B -> C -> A

When the traversal finds the edge from C to A, it will by default follow it. This is because we have not seen this edge yet. It will also visit vertex A again. This is because by default all vertices will be visited, regardless of whether already visited or not.

However, the traversal will not again following the outgoing edge from A to B. This is because we already have the edge from A to B in our current path.

These default settings will prevent infinite traversals.

To adjust the uniqueness for visiting vertices, there are the following options for uniqueness.vertices:

- "none": always visit a vertices, regardless of whether it was already visited or not

- "global": visit a vertex only if it was not visited in the traversal

- "path": visit a vertex if it is not included in the current path

To adjust the uniqueness for following edges, there are the following options for uniqueness.edges:

- "none": always follow an edge, regardless of whether it was followed before

- "global": follow an edge only if it wasn't followed in the traversal

- "path": follow an edge if it is not included in the current path

Note that uniqueness checking will have some effect on both runtime and memory usage. For example, when uniqueness checks are set to "global", arrays of visited vertices and edges must be kept in memory while the traversal is executed. Global uniqueness should thus only be used when a traversal is expected to visit few nodes.

In terms of runtime, turning off uniqueness checks (by setting both options to "none") is the best choice, but it is only safe for graphs that do not contain cycles. When uniqueness checks are deactivated in a graph with cycles, the traversal might not abort in a sensible amount of time.

Optimizations

There are a few options for making a traversal run faster.

The best option is to make the amount of visited vertices and followed edges as small as possible. This can be achieved by writing custom filter and expander functions. Such functions should only include vertices of interest, and only follow edges that might be interesting.

Traversal depth can also be bounded with the minDepth and maxDepth options.

Another way to speed up traversals is to write a custom visitor function. The default visitor function (trackingVisitor) will copy every visited vertex into the result. If vertices contain lots of data, this might be expensive. It is therefore recommended to only copy such data into the result that is actually needed. The default visitor function will also copy the full path to the visited document into the result. This is even more expensive and should be avoided if possible.

If the goal of a traversal is to only count the number of visited vertices, the prefab countingVisitor

will be much more efficient than the default visitor.

For graphs that are known to not contain any cycles, uniqueness checks should be turned off. This can achieved via the uniqueness configuration options. Note that uniqueness checks should not be turned off for graphs that are known contain cycles or if there is no information about the graph's structure.

By default, a traversal will only process a limited number of vertices. This is protect the user from unintentionally run a never-ending traversal on a graph with cyclic data. How many vertices will be processed at most is determined by the maxIterations configuration option. If a traversal hits the cap specified by maxIterations, it will abort and throw a too many iterations exception. If this error is encountered, the maxIterations value should be increased if it is made sure that the other traversal configuration parameters are sane and the traversal will abort naturally at some point.

Finally, the buildVertices configuration option can be set to false to avoid looking up and fully constructing vertex data. If all that's needed from vertices are the _id or _key attributes, the buildvertices option can be set to false. If visitor, filter or expandFilter functions need to access other vertex attributes, the option should not be changed.

Configuration Overview

This section summarizes the configuration attributes for the traversal object. The configuration can consist of the following attributes:

- visitor: visitor function for vertices. It will be called for all non-excluded vertices. The general visitor function signature is function (config, result, vertex, path). If the traversal order is preorder-expander, the connecting edges of the visited vertex will be passed as the fifth parameter, extending the function signature to: function (config, result, vertex, path, edges).

Visitor functions are not expected to return values, but they may modify the result variable as needed (e.g. by pushing vertex data into the result).

- expander: expander function that is responsible for returning edges and vertices directly connected to a vertex. The function signature is function (config, vertex, path). The expander function is required to return an array of connection objects, consisting of an edge and vertex attribute each. If there are no connecting edges, the expander is expected to return an empty array.

- filter: vertex filter function. The function signature is function (config, vertex, path). It may return one of the following values:

- undefined: vertex will be included in the result and connected edges will be traversed

- "exclude": vertex will not be included in the result and connected edges will be traversed

- "prune": vertex will be included in the result but connected edges will not be traversed

- [ "prune", "exclude" ]: vertex will not be included in the result and connected edges will not be returned

- expandFilter: filter function applied on each edge/vertex combination determined by the expander. The function signature is function (config, vertex, edge, path). The function should return true if the edge/vertex combination should be processed, and false if it should be ignored.

- sort: a filter function to determine the order in which connected edges are processed. The function signature is function (l, r). The function is required to return one of the following values:

- -1 if l should have a sort value less than r

- 1 if l should have a higher sort value than r

- 0 if l and r have the same sort value

- strategy: determines the visitation strategy. Possible values are depthfirst and breadthfirst.

- order: determines the visitation order. Possible values are preorder, postorder, and preorder-expander. preorder-expander is the same as preorder, except that the signature of the visitor function will change as described above.

- itemOrder: determines the order in which connections returned by the expander will be processed. Possible values are forward and backward.

- maxDepth: if set to a value greater than 0, this will limit the traversal to this maximum depth.

- minDepth: if set to a value greater than 0, all vertices found on a level below the minDepth level will not be included in the result.

- maxIterations: the maximum number of iterations that the traversal is allowed to perform. It is sensible to set this number so unbounded traversals will terminate at some point.

- uniqueness: an object that defines how repeated visitations of vertices should be handled. The uniqueness object can have a sub-attribute vertices, and a sub-attribute edges. Each sub-attribute can have one of the following values:

- "none": no uniqueness constraints

- "path": element is excluded if it is already contained in the current path. This setting may be sensible for graphs that contain cycles (e.g. A -> B -> C -> A).

- "global": element is excluded if it was already found/visited at any point during the traversal.

- buildVertices: this attribute controls whether vertices encountered during the traversal will be looked up in the database and will be made available to visitor, filter, and expandFilter functions. By default, vertices will be looked up and made available. However, there are some special use cases when fully constructing vertex objects is not necessary and can be avoided. For example, if a traversal is meant to only count the number of visited vertices but do not read any data from vertices, this option might be set to true.

Example Data

The following examples all use a vertex collection v and an edge collection e. The vertex collection v contains continents, countries, and capitals. The edge collection e contains connections between continents and countries, and between countries and capitals.

To set up the collections and populate them with initial data, the following script was used:

db._create("v");

db._createEdgeCollection("e");

// vertices: root node

db.v.save({ _key: "world", name: "World", type: "root" });

// vertices: continents

db.v.save({ _key: "continent-africa", name: "Africa", type: "continent" });

db.v.save({ _key: "continent-asia", name: "Asia", type: "continent" });

db.v.save({ _key: "continent-australia", name: "Australia", type: "continent" });

db.v.save({ _key: "continent-europe", name: "Europe", type: "continent" });

db.v.save({ _key: "continent-north-america", name: "North America", type: "continent" });

db.v.save({ _key: "continent-south-america", name: "South America", type: "continent" });

// vertices: countries

db.v.save({ _key: "country-afghanistan", name: "Afghanistan", type: "country", code: "AFG" });

db.v.save({ _key: "country-albania", name: "Albania", type: "country", code: "ALB" });

db.v.save({ _key: "country-algeria", name: "Algeria", type: "country", code: "DZA" });

db.v.save({ _key: "country-andorra", name: "Andorra", type: "country", code: "AND" });

db.v.save({ _key: "country-angola", name: "Angola", type: "country", code: "AGO" });

db.v.save({ _key: "country-antigua-and-barbuda", name: "Antigua and Barbuda", type: "country", code: "ATG" });

db.v.save({ _key: "country-argentina", name: "Argentina", type: "country", code: "ARG" });

db.v.save({ _key: "country-australia", name: "Australia", type: "country", code: "AUS" });

db.v.save({ _key: "country-austria", name: "Austria", type: "country", code: "AUT" });

db.v.save({ _key: "country-bahamas", name: "Bahamas", type: "country", code: "BHS" });

db.v.save({ _key: "country-bahrain", name: "Bahrain", type: "country", code: "BHR" });

db.v.save({ _key: "country-bangladesh", name: "Bangladesh", type: "country", code: "BGD" });

db.v.save({ _key: "country-barbados", name: "Barbados", type: "country", code: "BRB" });

db.v.save({ _key: "country-belgium", name: "Belgium", type: "country", code: "BEL" });

db.v.save({ _key: "country-bhutan", name: "Bhutan", type: "country", code: "BTN" });

db.v.save({ _key: "country-bolivia", name: "Bolivia", type: "country", code: "BOL" });

db.v.save({ _key: "country-bosnia-and-herzegovina", name: "Bosnia and Herzegovina", type: "country", code: "BIH" });

db.v.save({ _key: "country-botswana", name: "Botswana", type: "country", code: "BWA" });

db.v.save({ _key: "country-brazil", name: "Brazil", type: "country", code: "BRA" });

db.v.save({ _key: "country-brunei", name: "Brunei", type: "country", code: "BRN" });

db.v.save({ _key: "country-bulgaria", name: "Bulgaria", type: "country", code: "BGR" });

db.v.save({ _key: "country-burkina-faso", name: "Burkina Faso", type: "country", code: "BFA" });

db.v.save({ _key: "country-burundi", name: "Burundi", type: "country", code: "BDI" });

db.v.save({ _key: "country-cambodia", name: "Cambodia", type: "country", code: "KHM" });

db.v.save({ _key: "country-cameroon", name: "Cameroon", type: "country", code: "CMR" });

db.v.save({ _key: "country-canada", name: "Canada", type: "country", code: "CAN" });

db.v.save({ _key: "country-chad", name: "Chad", type: "country", code: "TCD" });

db.v.save({ _key: "country-chile", name: "Chile", type: "country", code: "CHL" });

db.v.save({ _key: "country-colombia", name: "Colombia", type: "country", code: "COL" });

db.v.save({ _key: "country-cote-d-ivoire", name: "Cote d'Ivoire", type: "country", code: "CIV" });

db.v.save({ _key: "country-croatia", name: "Croatia", type: "country", code: "HRV" });

db.v.save({ _key: "country-czech-republic", name: "Czech Republic", type: "country", code: "CZE" });

db.v.save({ _key: "country-denmark", name: "Denmark", type: "country", code: "DNK" });

db.v.save({ _key: "country-ecuador", name: "Ecuador", type: "country", code: "ECU" });

db.v.save({ _key: "country-egypt", name: "Egypt", type: "country", code: "EGY" });

db.v.save({ _key: "country-eritrea", name: "Eritrea", type: "country", code: "ERI" });

db.v.save({ _key: "country-finland", name: "Finland", type: "country", code: "FIN" });

db.v.save({ _key: "country-france", name: "France", type: "country", code: "FRA" });

db.v.save({ _key: "country-germany", name: "Germany", type: "country", code: "DEU" });

db.v.save({ _key: "country-people-s-republic-of-china", name: "People's Republic of China", type: "country", code: "CHN" });

// vertices: capitals

db.v.save({ _key: "capital-algiers", name: "Algiers", type: "capital" });

db.v.save({ _key: "capital-andorra-la-vella", name: "Andorra la Vella", type: "capital" });

db.v.save({ _key: "capital-asmara", name: "Asmara", type: "capital" });

db.v.save({ _key: "capital-bandar-seri-begawan", name: "Bandar Seri Begawan", type: "capital" });

db.v.save({ _key: "capital-beijing", name: "Beijing", type: "capital" });

db.v.save({ _key: "capital-berlin", name: "Berlin", type: "capital" });

db.v.save({ _key: "capital-bogota", name: "Bogota", type: "capital" });

db.v.save({ _key: "capital-brasilia", name: "Brasilia", type: "capital" });

db.v.save({ _key: "capital-bridgetown", name: "Bridgetown", type: "capital" });

db.v.save({ _key: "capital-brussels", name: "Brussels", type: "capital" });

db.v.save({ _key: "capital-buenos-aires", name: "Buenos Aires", type: "capital" });

db.v.save({ _key: "capital-bujumbura", name: "Bujumbura", type: "capital" });

db.v.save({ _key: "capital-cairo", name: "Cairo", type: "capital" });

db.v.save({ _key: "capital-canberra", name: "Canberra", type: "capital" });

db.v.save({ _key: "capital-copenhagen", name: "Copenhagen", type: "capital" });

db.v.save({ _key: "capital-dhaka", name: "Dhaka", type: "capital" });

db.v.save({ _key: "capital-gaborone", name: "Gaborone", type: "capital" });

db.v.save({ _key: "capital-helsinki", name: "Helsinki", type: "capital" });

db.v.save({ _key: "capital-kabul", name: "Kabul", type: "capital" });

db.v.save({ _key: "capital-la-paz", name: "La Paz", type: "capital" });

db.v.save({ _key: "capital-luanda", name: "Luanda", type: "capital" });

db.v.save({ _key: "capital-manama", name: "Manama", type: "capital" });

db.v.save({ _key: "capital-nassau", name: "Nassau", type: "capital" });

db.v.save({ _key: "capital-n-djamena", name: "N'Djamena", type: "capital" });

db.v.save({ _key: "capital-ottawa", name: "Ottawa", type: "capital" });

db.v.save({ _key: "capital-ouagadougou", name: "Ouagadougou", type: "capital" });

db.v.save({ _key: "capital-paris", name: "Paris", type: "capital" });

db.v.save({ _key: "capital-phnom-penh", name: "Phnom Penh", type: "capital" });

db.v.save({ _key: "capital-prague", name: "Prague", type: "capital" });

db.v.save({ _key: "capital-quito", name: "Quito", type: "capital" });

db.v.save({ _key: "capital-saint-john-s", name: "Saint John's", type: "capital" });

db.v.save({ _key: "capital-santiago", name: "Santiago", type: "capital" });

db.v.save({ _key: "capital-sarajevo", name: "Sarajevo", type: "capital" });

db.v.save({ _key: "capital-sofia", name: "Sofia", type: "capital" });

db.v.save({ _key: "capital-thimphu", name: "Thimphu", type: "capital" });

db.v.save({ _key: "capital-tirana", name: "Tirana", type: "capital" });

db.v.save({ _key: "capital-vienna", name: "Vienna", type: "capital" });

db.v.save({ _key: "capital-yamoussoukro", name: "Yamoussoukro", type: "capital" });

db.v.save({ _key: "capital-yaounde", name: "Yaounde", type: "capital" });

db.v.save({ _key: "capital-zagreb", name: "Zagreb", type: "capital" });

// edges: continent -> world

db.e.save("v/continent-africa", "v/world", { type: "is-in" });

db.e.save("v/continent-asia", "v/world", { type: "is-in" });

db.e.save("v/continent-australia", "v/world", { type: "is-in" });

db.e.save("v/continent-europe", "v/world", { type: "is-in" });

db.e.save("v/continent-north-america", "v/world", { type: "is-in" });

db.e.save("v/continent-south-america", "v/world", { type: "is-in" });

// edges: country -> continent

db.e.save("v/country-afghanistan", "v/continent-asia", { type: "is-in" });

db.e.save("v/country-albania", "v/continent-europe", { type: "is-in" });

db.e.save("v/country-algeria", "v/continent-africa", { type: "is-in" });

db.e.save("v/country-andorra", "v/continent-europe", { type: "is-in" });

db.e.save("v/country-angola", "v/continent-africa", { type: "is-in" });

db.e.save("v/country-antigua-and-barbuda", "v/continent-north-america", { type: "is-in" });

db.e.save("v/country-argentina", "v/continent-south-america", { type: "is-in" });

db.e.save("v/country-australia", "v/continent-australia", { type: "is-in" });

db.e.save("v/country-austria", "v/continent-europe", { type: "is-in" });

db.e.save("v/country-bahamas", "v/continent-north-america", { type: "is-in" });

db.e.save("v/country-bahrain", "v/continent-asia", { type: "is-in" });

db.e.save("v/country-bangladesh", "v/continent-asia", { type: "is-in" });

db.e.save("v/country-barbados", "v/continent-north-america", { type: "is-in" });

db.e.save("v/country-belgium", "v/continent-europe", { type: "is-in" });

db.e.save("v/country-bhutan", "v/continent-asia", { type: "is-in" });

db.e.save("v/country-bolivia", "v/continent-south-america", { type: "is-in" });

db.e.save("v/country-bosnia-and-herzegovina", "v/continent-europe", { type: "is-in" });

db.e.save("v/country-botswana", "v/continent-africa", { type: "is-in" });

db.e.save("v/country-brazil", "v/continent-south-america", { type: "is-in" });

db.e.save("v/country-brunei", "v/continent-asia", { type: "is-in" });

db.e.save("v/country-bulgaria", "v/continent-europe", { type: "is-in" });

db.e.save("v/country-burkina-faso", "v/continent-africa", { type: "is-in" });

db.e.save("v/country-burundi", "v/continent-africa", { type: "is-in" });

db.e.save("v/country-cambodia", "v/continent-asia", { type: "is-in" });

db.e.save("v/country-cameroon", "v/continent-africa", { type: "is-in" });

db.e.save("v/country-canada", "v/continent-north-america", { type: "is-in" });

db.e.save("v/country-chad", "v/continent-africa", { type: "is-in" });

db.e.save("v/country-chile", "v/continent-south-america", { type: "is-in" });

db.e.save("v/country-colombia", "v/continent-south-america", { type: "is-in" });

db.e.save("v/country-cote-d-ivoire", "v/continent-africa", { type: "is-in" });

db.e.save("v/country-croatia", "v/continent-europe", { type: "is-in" });

db.e.save("v/country-czech-republic", "v/continent-europe", { type: "is-in" });

db.e.save("v/country-denmark", "v/continent-europe", { type: "is-in" });

db.e.save("v/country-ecuador", "v/continent-south-america", { type: "is-in" });

db.e.save("v/country-egypt", "v/continent-africa", { type: "is-in" });

db.e.save("v/country-eritrea", "v/continent-africa", { type: "is-in" });

db.e.save("v/country-finland", "v/continent-europe", { type: "is-in" });

db.e.save("v/country-france", "v/continent-europe", { type: "is-in" });

db.e.save("v/country-germany", "v/continent-europe", { type: "is-in" });

db.e.save("v/country-people-s-republic-of-china", "v/continent-asia", { type: "is-in" });

// edges: capital -> country

db.e.save("v/capital-algiers", "v/country-algeria", { type: "is-in" });

db.e.save("v/capital-andorra-la-vella", "v/country-andorra", { type: "is-in" });

db.e.save("v/capital-asmara", "v/country-eritrea", { type: "is-in" });

db.e.save("v/capital-bandar-seri-begawan", "v/country-brunei", { type: "is-in" });

db.e.save("v/capital-beijing", "v/country-people-s-republic-of-china", { type: "is-in" });

db.e.save("v/capital-berlin", "v/country-germany", { type: "is-in" });

db.e.save("v/capital-bogota", "v/country-colombia", { type: "is-in" });

db.e.save("v/capital-brasilia", "v/country-brazil", { type: "is-in" });

db.e.save("v/capital-bridgetown", "v/country-barbados", { type: "is-in" });

db.e.save("v/capital-brussels", "v/country-belgium", { type: "is-in" });

db.e.save("v/capital-buenos-aires", "v/country-argentina", { type: "is-in" });

db.e.save("v/capital-bujumbura", "v/country-burundi", { type: "is-in" });

db.e.save("v/capital-cairo", "v/country-egypt", { type: "is-in" });

db.e.save("v/capital-canberra", "v/country-australia", { type: "is-in" });

db.e.save("v/capital-copenhagen", "v/country-denmark", { type: "is-in" });

db.e.save("v/capital-dhaka", "v/country-bangladesh", { type: "is-in" });

db.e.save("v/capital-gaborone", "v/country-botswana", { type: "is-in" });

db.e.save("v/capital-helsinki", "v/country-finland", { type: "is-in" });

db.e.save("v/capital-kabul", "v/country-afghanistan", { type: "is-in" });

db.e.save("v/capital-la-paz", "v/country-bolivia", { type: "is-in" });

db.e.save("v/capital-luanda", "v/country-angola", { type: "is-in" });

db.e.save("v/capital-manama", "v/country-bahrain", { type: "is-in" });

db.e.save("v/capital-nassau", "v/country-bahamas", { type: "is-in" });

db.e.save("v/capital-n-djamena", "v/country-chad", { type: "is-in" });

db.e.save("v/capital-ottawa", "v/country-canada", { type: "is-in" });

db.e.save("v/capital-ouagadougou", "v/country-burkina-faso", { type: "is-in" });

db.e.save("v/capital-paris", "v/country-france", { type: "is-in" });

db.e.save("v/capital-phnom-penh", "v/country-cambodia", { type: "is-in" });

db.e.save("v/capital-prague", "v/country-czech-republic", { type: "is-in" });

db.e.save("v/capital-quito", "v/country-ecuador", { type: "is-in" });

db.e.save("v/capital-saint-john-s", "v/country-antigua-and-barbuda", { type: "is-in" });

db.e.save("v/capital-santiago", "v/country-chile", { type: "is-in" });

db.e.save("v/capital-sarajevo", "v/country-bosnia-and-herzegovina", { type: "is-in" });

db.e.save("v/capital-sofia", "v/country-bulgaria", { type: "is-in" });

db.e.save("v/capital-thimphu", "v/country-bhutan", { type: "is-in" });

db.e.save("v/capital-tirana", "v/country-albania", { type: "is-in" });

db.e.save("v/capital-vienna", "v/country-austria", { type: "is-in" });

db.e.save("v/capital-yamoussoukro", "v/country-cote-d-ivoire", { type: "is-in" });

db.e.save("v/capital-yaounde", "v/country-cameroon", { type: "is-in" });

db.e.save("v/capital-zagreb", "v/country-croatia", { type: "is-in" });

Edges, Identifiers, Handles

This is an introduction to ArangoDB's interface for edges and how to handle edges from the JavaScript shell arangosh. For other languages see the corresponding language API.

Edges in ArangoDB are special documents. In addition to the internal attributes _key, _id and _rev, they have two attributes _from and _to, which contain document handles, namely the start-point and the end-point of the edge. The values of _from and _to are immutable once saved.

Edge collections are special collections that store edge documents. Edge documents are connection documents that reference other documents. The type of a collection must be specified when a collection is created and cannot be changed afterwards.

To change edge endpoints you would need to remove old document/edge and insert new one. Other fields can be updated as in default collection.

Working with Edges

Insert

edge-collection.insert(from, to, document)

Saves a new edge and returns the document-handle. from and to must be documents or document references.

edge-collection.insert(from, to, document, waitForSync)

The optional waitForSync parameter can be used to force synchronization of the document creation operation to disk even in case that the waitForSync flag had been disabled for the entire collection. Thus, the waitForSync parameter can be used to force synchronization of just specific operations. To use this, set the waitForSync parameter to true. If the waitForSync parameter is not specified or set to false, then the collection's default waitForSync behavior is applied. The waitForSync parameter cannot be used to disable synchronization for collections that have a default waitForSync value of true.

Examples

arangosh> db._create("vertex");

arangosh> db._createEdgeCollection("relation");

arangosh> v1 = db.vertex.insert({ name : "vertex 1" });

arangosh> v2 = db.vertex.insert({ name : "vertex 2" });

arangosh> e1 = db.relation.insert(v1, v2, { label : "knows" });

arangosh> db._document(e1);

show execution results

Edges

edge-collection.edges(vertex)

The edges operator finds all edges starting from (outbound) or ending in (inbound) vertex.

edge-collection.edges(vertices)

The edges operator finds all edges starting from (outbound) or ending in (inbound) a document from vertices, which must a list of documents or document handles.

arangosh> db._create("vertex");

arangosh> db._createEdgeCollection("relation");

arangosh> myGraph.v1 = db.vertex.insert({ name : "vertex 1" });

arangosh> myGraph.v2 = db.vertex.insert({ name : "vertex 2" });

arangosh> myGraph.e1 = db.relation.insert(myGraph.v1, myGraph.v2, { label : "knows" });

arangosh> db._document(myGraph.e1);

arangosh> db.relation.edges(myGraph.e1._id);

show execution results

InEdges

edge-collection.inEdges(vertex)

The edges operator finds all edges ending in (inbound) vertex.

edge-collection.inEdges(vertices)

The edges operator finds all edges ending in (inbound) a document from vertices, which must a list of documents or document handles.

Examples

arangosh> db._create("vertex");

arangosh> db._createEdgeCollection("relation");

arangosh> myGraph.v1 = db.vertex.insert({ name : "vertex 1" });

arangosh> myGraph.v2 = db.vertex.insert({ name : "vertex 2" });

arangosh> myGraph.e1 = db.relation.insert(myGraph.v1, myGraph.v2, { label : "knows" });

arangosh> db._document(myGraph.e1);

arangosh> db.relation.inEdges(myGraph.v1._id);

arangosh> db.relation.inEdges(myGraph.v2._id);

show execution results

OutEdges

edge-collection.outEdges(vertex)

The edges operator finds all edges starting from (outbound) vertices.

edge-collection.outEdges(vertices)

The edges operator finds all edges starting from (outbound) a document from vertices, which must a list of documents or document handles.

Examples

arangosh> db._create("vertex");

arangosh> db._createEdgeCollection("relation");

arangosh> myGraph.v1 = db.vertex.insert({ name : "vertex 1" });

arangosh> myGraph.v2 = db.vertex.insert({ name : "vertex 2" });

arangosh> myGraph.e1 = db.relation.insert(myGraph.v1, myGraph.v2, { label : "knows" });

arangosh> db._document(myGraph.e1);

arangosh> db.relation.outEdges(myGraph.v1._id);

arangosh> db.relation.outEdges(myGraph.v2._id);

show execution results

Foxx

Foxx is a framework that allows you to write data-centric microservices. It is executed directly inside of ArangoDB and grants you raw access to your data. With the framework you can embed complex queries and data-focused business logic into your database.

Using Foxx will give you several benefits above writing the code in separate applications. First of all it yields performance benefits, the data analysis and modification is executed as close to the data as possible. Also it requires one network hop less as the Foxx has direct access to the data and does not have to query a database located on a different server. Second your application is completely free of any database related information like queries, it just requests an HTTP-endpoint that gives the data in the expected format. This also allows to reuse this endpoint in different applications without copying the queries. Third Foxx adds another level of security to your application, you can manage the access rights on any level you like: application, collection, document or even attribute. Finally Foxx is automatically scaled alongside your database: if the number of ArangoDB servers grows so does your Foxx servers, both have their bottleneck at the same point.

In the recent movements of software development many companies of all sizes talk about moving away from large monolithic applications and switch over to a microservice based software architecture. Because microservices yield many benefits in terms of scaling and zero-downtime.

During the design of Foxx we followed the concepts and ideas of microservices. Our baseline is the definition by Martin Fowler and James Lewis given in their microservice article. The following table lists a mapping of microservice concepts and how they can be realized within Foxx:

microservice

Foxx

description

Few lines of Code

Little Boilerplate

Most of the boilerplate is already handled by the Foxx environment. You can focus on your business logic.

HTTP API

Controller

The Foxx controller allows to easily define HTTP API routes.

Independently Deployable

Foxx Manager

Foxxes run in ArangoDB Coordinators. These are independent from one another. Foxxes can be installed on an arbitrary number of these Coordinators.

Access other Microservices

Requests

The Foxx request module offers a simple functionality to contact any other API via HTTP.

Scalability

Sharding

The dataset managed by a Foxx micro-service might be too large for a single server. The underlying ArangoDB infrastructure can handle this for you.

Persistence

Repositories

Foxx is part of a multi-model database. It has all features to persist data of diverse formats (Graphs, Documents) built-in.

Reuse Services

Arango Store

The Arango Store contains several Foxxes. Many of them are general-use services that are required in many applications. You can just reuse the reviewed and tested code of other Foxx users.

Automated Deployment

Foxx Manager

The Foxx manager tool allows to trigger installation of new applications. Also there is a native system HTTP API offering this feature. You can use either one as a step in your build automation tool.

Design for failure

Coordinators

ArangoDBs cluster setup is designed for failure. You can make use of this feature in your Foxxes.

Different Languages

Connect to other APIs

Foxxes are only implemented in JavaScript. However not your entire application has to be written in Foxx. You can connect via HTTP to services in other languages.

Different Databases

Multi-Model

In most setups you need different database technologies because you have several data formats. ArangoDB is a multi-model database that serves many formats. However if you still need another database you can connect to it via HTTP.

Asynchronous Calls

Synchronous with multiple contexts

Foxx only allows synchronous execution. However it is executed in a multi-threaded fashion, hence blocking one thread is not a problem. Also background tasks are available for long-running tasks that should be executed outside of the request-response cycle.

Security

Authentication and Session

Foxx offers internal APIs to handle authentication and manage session data. It is just a handful lines of code and your service is secured.

Protected API

Libraries

Sometimes you have sensitive data that is not allowed to ever leave your database. Using libraries in Foxx you can write internal APIs that can only be accessed from the same server and are not exposed to the outside world. Perfect to keep your data inside but still using the paradigm of a microservice for it.

How To

Recipes

In ArangoDBs cookbook there are several recipes on how to design a microservice based application with the help of Foxx.

Maintenance UI

Built-in Web server

ArangoDB has a built-in Web server, allowing Foxxes to ship a Webpage as their UI. So you can attach a maintenance UI directly to your microservice and have it deployed with it in the same step.

Easy Setup

ArangoDB on Mesos

ArangoDB perfectly integrates with the [Mesos project](http://mesos.apache.org/), a distributed kernel written for data centers. You can set up and deploy a Mesos instance running ArangoDB and having your microservice on top of it with just a single command.

Maintenance

Marathon

If you are running ArangoDB on top of Mesos you can use [Marathon](https://mesosphere.github.io/marathon/) to keep all your instances alive and scale up or down with just a few clicks. Do not bother with long configuration, that is all handled by the underlying systems.

As you can see most concepts of microservices are natively available in Foxx plus several additions that ease your development process. Thats why we like to call Foxx a "microservice framework": It allows you to write and deploy data-centric microservices on top of ArangoDB.

If this short introduction has piqued your interest you can find more details in the sections below. We recommend to start with Foxx in a nutshell which gives a really quick introduction in how to create and use your first Foxx. All other sections will give very detailed informations about the features available and a peak behind the curtains.

Foxx in a nutshell

This chapter is for the impatient and for the ones that just want to get a quick overview of what Foxx is, how it works and how to get it up and running in short time. It is tutorial that should be solvable in 10 minutes or less and will guide you to your first application giving a few insights on the way.

Read More

Install a Foxx Application

You would like to get some inspiration from running Applications? Or you already have an Application that does what you need? This chapter will guide you to get an Application up and running.

Read More

Develop your own Foxx

Ready to get to bare metal? In this chapter you will get to know the details on Foxx development. It will explain how to use the development mode and which debugging mechanisms are at hand. It introduces the tips and tricks on how to build your own API. And presents all libraries that are available from a Foxx Application.

Read More

Running Foxx Applications in production

This chapter is dedicated to all the possibilities on how to run a Foxx Application in production. It will start with how to leverage it from development to production. We will continue with migration of the Application to a newer version.

Read More

Foxx in a cluster

Ready to scale out? Want to setup your micro-service based application? This chapter will explain what has to be changed to run a Foxx application within a cluster setup. It will also explain how to move your application from a single server to an unlimited amount of servers.

Read More

Foxx in a nutshell

Quick navigation

- Quick start

- Create a new Application

- Interactive Documentation

- Developing in Foxx

- Foxx folder structure

- List content of a collection

- Create a new document

- Read a single document

- Update a document

- Delete a document

- Execute a query

- Conclusion

</a>

Quick Start

You can follow this tutorial using ArangoDBs web interface.

Start the interface from http://localhost:8529 (default configuration) and click on the Applications tab on the top.

Note

Before we start: Please Install and run ArangoDB in version 2.5 or newer

Create a new application

1. Click on  Add Application .

2. In the New App tab, you will be offered a dialog to fill in some information about your application.

3. For this tutorial we will use the following information:

- Author: ArangoDB

- Name: firstApp

- Description: This is my first Foxx app

- License: Apache 2

- Collections: firstCollection

4. Click Generate

Some details you should know:

- The information entered here is basically meta information that will be displayed in the web interface.

- Collections  will be created for you and are local to your Foxx application. For all collections CRUD endpoints will be generated for you.

- You can define up to 8 collections. But for this tutorial stick to exactly one.

Interactive documentation

Now you should see your new application in the list of installed applications. Clicking on it will lead you to the details page. This page provides the meta information about the application entered in the generating dialog. You can modify this information in the applications manifest file. Additionally, the interactive swagger-documentation could be used to test the app. Whenever we modify something in the app during this tutorial it should be visible in this documentation. So you can directly test if your modification has worked or not.

Developing in Foxx

In order to develop your Foxx you now have two options. If you do not have access to the file-system of your ArangoDB instance you can only pick option 1).

1. Modify the files locally.

- Download the application's source in a zip archive to your local file-system by clicking on the download button.

- Extracted the file.

- Later in this tutorial we will modify the files.

- After this modifications you have to repack the sources into a zip file.

- You can then click on  upgrade , select  Zip  and upload your zip file.

- This will replace the Foxx stored in ArangoDB by your own implementation.

2. Use Foxx development mode.

- Activate the development mode by clicking on  Set Dev .

- The Web Interface will print the folder containing the sources of your Application.

- Open this folder to modify the files.

- All changes to these files will be applied directly as long as the development mode is active.

- Deactivate the development mode by clicking on  Set Pro .

App folder structure

If you have opened the folder containing your App the folder structure should be the following:

unix>tree app

app

├── controllers

│   └── firstCollection.js

├── manifest.json

├── models

│   └── FirstCollection.js

├── repositories

│   └── firstCollection.js

└── scripts

├── setup.js

└── teardown.js

4 directories, 6 files

- controllers  define endpoints for your requests.

- repositories  define collections used by this application and allow to attach custom code to them.

- models  define object models to be stored in collections. This allows to execute object validation before storing them in your database.

- scripts  contain code that will be executed on specific events. E.g. setup on application install and teardown on application uninstall.

- manifest.json  contains meta information and tells ArangoDB where to find the files for this application. See in the second screenshot.

For this tutorial we will only work with the file:  controllers/firstCollection.js  Now open your favorite editor and have a deeper look into the code.

Note

Most of the code described in this tutorial is already generated

List content of a collection

The following route lists all elements of our collection:

GET http://localhost:8529/_db/_system/firstApp/firstCollection/firstCollection

It is defined in the source in the following way:

/** Lists of all FirstCollection

*

* This function simply returns the list of all FirstCollection.

*/

controller.get('/firstCollection', function (req, res) {

res.json(_.map(FirstCollection_repo.all(), function (model) {

return model.forClient();

}));

});

Some details you should know about the code:

- The  controller  gives you options to create routes.

- controller.get(path, callback)  creates a  GET  request handler on  path . It will execute  callback  whenever triggered.

- FirstCollection_repo  is an instance of firstCollection

- _.map(collection.all(), function())  transforms all documents in the collection to readable JSON.

- res.json()  creates a JSON response for the client.

- The comment above the function will generate a nice documentation.

Create a new document

This code-snippet defines the route to create a new document in our collection:

POST http://localhost:8529/_db/_system/firstApp/firstCollection/firstCollection

/** Creates a new FirstCollection

*

* Creates a new FirstCollection-Item. The information has to be in the

* requestBody.

*/

controller.post('/firstCollection', function (req, res) {

var firstCollection = req.params('firstCollection');

res.json(FirstCollection_repo.save(firstCollection).forClient());

}).bodyParam('firstCollection', 'The FirstCollection you want to create', FirstCollection);

This route is used to store JSON objects following a strict pattern into the collection. The pattern is defined by the Model given in the file  models/FirstCollection.js :

(function () {

'use strict';

var Foxx = require('org/arangodb/foxx'),

Joi = require('joi'),

Model;

Model = Foxx.Model.extend({

schema: {

// Describe the attributes with Joi here

'_key': Joi.string(),

}

});

exports.Model = Model;

}());

This defines that only documents having a  _key  can be stored and nothing else. Now lets also allow to store an optional  name  by changing the following lines:

Model = Foxx.Model.extend({

schema: {

// Describe the attributes with Joi here

'_key': Joi.string(),

'name': Joi.string().optional(),

}

});

To test it you can for example store the following object. Note the _key is the internal identifier for this object and can either be user-defined or will be generated otherwise. In this tutorial we will refer to this key again.

{

"_key": "123456",

"name": "Alice"

}

To test the route you can copy & paste this object into the interactive documentation or execute the following line in your shell:

unix> curl -X POST http://localhost:8529/_db/_system/firstApp/firstCollection/firstCollection --data '{"_key":"123456", "name":"Alice"}'

{"_key":"123456","name":"Alice"}

You can also try to store documents having different attributes or not having name.

Some details you should know about the code:

- controller.post(path, callback)  creates a  POST  request handler on  path . It will execute  callback  whenever triggered.

- req.params()  allows to access all parameters, including the body, sent with the request.

- FirstCollection_repo.save()  will store a JSON document into our collection

- forClient()  will create JSON output for the client

- bodyParam  defines that there is an body expected in each request and the body should correspond to a valid FirstCollection object. All other bodies will be rejected and a documentation will be generated for this body. It can be accessed in the request parameters via it's name 'firstCollection'

Read a single document

This route allows to read a specific document in our collection, identified by Id:

GET http://localhost:8529/_db/_system/firstApp/firstCollection/firstCollection/:id

/** Reads a FirstCollection

*

* Reads a FirstCollection-Item.

*/

controller.get('/firstCollection/:id', function (req, res) {

var id = req.params('id');

res.json(FirstCollection_repo.byId(id).forClient());

}).pathParam('id', {

description: 'The id of the FirstCollection-Item',

type: 'string'

});

This route now allows us to read exactly one specific document stored in our collection. In the last route we have inserted the "Alice" document, now we want to get that one back. In order to achieve this we have to replace the "_id" with the "_key" we have stored earlier. In the interactive documentation there is a text field to enter the value, or you can execute the following CURL command:

unix> curl -X GET http://localhost:8529/_db/_system/firstApp/firstCollection/firstCollection/123456

{"_key":"123456","name":"Alice"}

- controller.get(path, callback)  creates a  GET  request handler on  path . It will execute  callback  whenever triggered.

- The  :id  in the path defines that this part of the route will be interpreted as a parameter with the name id.

- byId  fetches a document from the collection by it's unique identifier:  _key

- pathParam  documents the path parameter  id  as described in the path. It also restricts it to be a string.

Update a document

This code shows how to update a specific document in our collection. The API call is:

GET http://localhost:8529/_db/_system/firstApp/firstCollection/firstCollection/:id

/** Updates a FirstCollection

*

* Changes a FirstCollection-Item. The information has to be in the

* requestBody.

*/

controller.put('/firstCollection/:id', function (req, res) {

var id = req.params('id'),

firstCollection = req.params('firstCollection');

res.json(FirstCollection_repo.replaceById(id, firstCollection));

}).pathParam('id', {

description: 'The id of the FirstCollection-Item',

type: 'string'

}).bodyParam('firstCollection', 'The FirstCollection you want your old one to be replaced with', FirstCollection);

This route allows us to replace one specific document. To test it let's replace "Alice" with "Bob" by filling the input field in the documentation with:

{

"name": "Bob"

}

Or sending the following CURL request:

unix> curl -X PUT http://localhost:8529/_db/_system/firstApp/firstCollection/firstCollection/123456 --data '{"name": "Bob"}'

{"_id":"firstApp_firstCollection/123456","_rev":"13346908308","_oldRev":"13267150996","_key":"123456"}

Note here: The  _rev and  _oldRev  values are generated by ArangoDB and can be used to validate if cached versions of the document are still up-to-date

Now let us read the document again using the  GET  request from before. It shall return "Bob" now:

unix> curl -X GET http://localhost:8529/_db/_system/firstApp/firstCollection/firstCollection/123456

{"_key":"123456","name":"Bob"}

Some details you should now about the code:

- controller.put(path, callback)  creates a  PUT  request handler on  path . It will execute  callback  whenever triggered.

- replaceById  overwrites a document from the collection with a newer version.

Delete a document

This call allows to delete a specific document in our collection:

DELETE http://localhost:8529/_db/_system/firstApp/firstCollection/firstCollection/:id

The corresponding controller function is just a few lines:

/** Removes a FirstCollection

*

* Removes a FirstCollection-Item.

*/

controller.del('/firstCollection/:id', function (req, res) {

var id = req.params('id');

FirstCollection_repo.removeById(id);

res.json({ success: true });

}).pathParam('id', {

description: 'The ID of the FirstCollection-Item',

type: 'string'

}).errorResponse(ArangoError, 404, 'The document could not be found');

Let's also test this route:

unix> curl -X DELETE http://localhost:8529/_db/_system/firstApp/firstCollection/firstCollection/123456

{"success":true}

unix> curl -X GET http://localhost:8529/_db/_system/firstApp/firstCollection/firstCollection/123456

{"error":"document not found"}

The document is gone now, if you are using development mode you will also receive the stacktrace for the error.

Some details you should now about the code:

- controller.del(path, callback)  creates a  DELETE  request handler on  path . It will execute  callback  whenever triggered.

- removeById  will delete a document from the collection.

- errorResponse  defines the message that should be delivered to the client whenever there was an error. In this case the user will get a classical  404 .

- If no  errorResponse  is given a internal  500  will be returned by the server.

Adding a new route / execute a query

We now want to add some functionality, a simple search query. First we create an endpoint that allows to execute a simple AQL query. The query will search for all documents having a specific  name  attribute.

Note

This part of the code is not generated. You can just add it after all the other routes.

GET http://localhost:8529/_db/_system/firstApp/firstCollection/byName/:name

var db = require('org/arangodb').db;

var searchQuery = 'FOR x IN @@collection FILTER x.name == @name RETURN x';

/** Search a FirstCollection by name

*

* Searches for a FirstCollection-Item

* with a specific name attribute.

*/

controller.get('/firstCollection/byName/:name', function (req, res) {

res.json(

db._query(searchQuery, {

'@collection': FirstCollection_repo.collection.name(),

'name': req.params('name')

}).toArray()

);

}).pathParam('name', {

type: joi.string().required().description('The value to search for')

});

This query searches for all documents that have the given name attribute. In order to test it let us install several instances of "Alice" in the POST route:

{

"name": "Alice"

}

unix> curl -X POST http://localhost:8529/_db/_system/firstApp/firstCollection/firstCollection --data '{"name":"Alice"}'

unix> curl -X POST http://localhost:8529/_db/_system/firstApp/firstCollection/firstCollection --data '{"name":"Alice"}'

unix> curl -X POST http://localhost:8529/_db/_system/firstApp/firstCollection/firstCollection --data '{"name":"Alice"}'

unix> curl -X POST http://localhost:8529/_db/_system/firstApp/firstCollection/firstCollection --data '{"name":"Alice"}'

unix> curl -X POST http://localhost:8529/_db/_system/firstApp/firstCollection/firstCollection --data '{"name":"Alice"}'

unix> curl -X POST http://localhost:8529/_db/_system/firstApp/firstCollection/firstCollection --data '{"name":"Alice"}'

And now let's issue the query with  Alice

unix> curl -X GET http://localhost:8529/_db/_system/firstApp/firstCollection/firstCollection/byName/Alice | json_pp

[

{

"_key" : "15352375444",

"_rev" : "15352375444",

"name" : "Alice",

"_id" : "firstApp_firstCollection/15352375444"

},

{

"_key" : "15346346132",

"_rev" : "15346346132",

"name" : "Alice",

"_id" : "firstApp_firstCollection/15346346132"

},

{

"name" : "Alice",

"_id" : "firstApp_firstCollection/15344380052",

"_rev" : "15344380052",

"_key" : "15344380052"

},

{

"_key" : "15341824148",

"_rev" : "15341824148",

"name" : "Alice",

"_id" : "firstApp_firstCollection/15341824148"

},

{

"_key" : "15350343828",

"_rev" : "15350343828",

"name" : "Alice",

"_id" : "firstApp_firstCollection/15350343828"

},

{

"_key" : "15348377748",

"_rev" : "15348377748",

"_id" : "firstApp_firstCollection/15348377748",

"name" : "Alice"

}

]

Some details you should know about the code:

- db  and  searchQuery  are global variables available in all routes. Be careful: Foxx is multithreaded, writing to global variables will not be propagated to other threads. If you need shared information store it in a collection.

- db._query().toArray()  executes an AQL query on the server and evaluates it directly to an array.

- @collection  is a bindParameter for collections and will be replaced in the query.

- name  is a bindParameter for a variable value and will be replaced in the query.

Conclusion

Now we have managed to create a very basic CRUD based Foxx. It is a REST-based wrapper around the interface of one of ArangoDBs collections and you can now start to include custom functionality into it. The first custom function we have created already is the byName query. It has stored an AQL on the server-side keeping your application clean of it. Now you can also access this query from several applications increasing the re-usability of code.

Installing Foxx Applications

Introduction to Foxx manager

The Foxx manager is a shell program. It should have been installed under /usr/bin or /usr/local/bin when installing the ArangoDB package. An instance of the ArangoDB server must be up and running.

unix> foxx-manager

Expecting a command, please try:

Example usage:

foxx-manager install <app-info> <mount-point> option1=value1

foxx-manager uninstall <mount-point>

Further help:

foxx-manager help   for the list of foxx-manager commands

foxx-manager --help for the list of options

The most important commands are

- install: Installs a Foxx application to a local URL and calls its setup method

- uninstall: Deletes a Foxx application and calls its teardown method

- list: Lists all installed Foxx applications (alias: installed)

When dealing with a fresh install of ArangoDB, there should be no installed applications besides the system applications that are shipped with ArangoDB.

unix> foxx-manager list

Mount                   Name           Author               Description                                       Version    Development

---------------------   ------------   ------------------   -----------------------------------------------   --------   ------------

/_admin/aardvark        aardvark       ArangoDB GmbH        ArangoDB Admin Web Interface                      1.0        no

/_api/gharial           gharial        ArangoDB GmbH        ArangoDB Graph Module                             0.1        no

/_system/cerberus       cerberus       ArangoDB GmbH        Password Manager                                  0.0.1      no

/_system/sessions       sessions       ArangoDB GmbH        Session storage for Foxx.                         0.1        no

/_system/simple-auth    simple-auth    ArangoDB GmbH        Simple password-based authentication for Foxx.    0.1        no

---------------------   ------------   ------------------   -----------------------------------------------   --------   ------------

5 application(s) found

There are currently several applications installed, all of them are system applications. You can safely ignore system applications.

We are now going to install the hello world application. It is called "hello-foxx" - no surprise there.

unix> foxx-manager install hello-foxx /example

Application hello-foxx version 1.5.0 installed successfully at mount point /example

The second parameter /example is the mount path of the application. Assuming that you are running a local installation of ArangoDB you should be able to access the example application under

http://localhost:8529/_db/_system/example

using your favorite browser. If you are connecting to a remote instance please replace  localhost  accordingly. It will now also be visible when using the list command.

unix> foxx-manager list

Mount                   Name           Author               Description                                       Version    Development

---------------------   ------------   ------------------   -----------------------------------------------   --------   ------------

/_admin/aardvark        aardvark       ArangoDB GmbH        ArangoDB Admin Web Interface                      1.0        no

/_api/gharial           gharial        ArangoDB GmbH        ArangoDB Graph Module                             0.1        no

/_system/cerberus       cerberus       ArangoDB GmbH        Password Manager                                  0.0.1      no

/_system/sessions       sessions       ArangoDB GmbH        Session storage for Foxx.                         0.1        no

/_system/simple-auth    simple-auth    ArangoDB GmbH        Simple password-based authentication for Foxx.    0.1        no

/example                hello-foxx     Frank Celler         This is 'Hello World' for ArangoDB Foxx.          1.5.0      no

---------------------   ------------   ------------------   -----------------------------------------------   --------   ------------

6 application(s) found

You can install the application again under different mount path.

unix> foxx-manager install hello-foxx /hello

Application hello-foxx version 1.5.0 installed successfully at mount point /hello

You now have two separate instances of the same application. They are completely independent of each other.

unix> foxx-manager list

Mount                   Name           Author               Description                                       Version    Development

---------------------   ------------   ------------------   -----------------------------------------------   --------   ------------

/_admin/aardvark        aardvark       ArangoDB GmbH        ArangoDB Admin Web Interface                      1.0        no

/_api/gharial           gharial        ArangoDB GmbH        ArangoDB Graph Module                             0.1        no

/_system/cerberus       cerberus       ArangoDB GmbH        Password Manager                                  0.0.1      no

/_system/sessions       sessions       ArangoDB GmbH        Session storage for Foxx.                         0.1        no

/_system/simple-auth    simple-auth    ArangoDB GmbH        Simple password-based authentication for Foxx.    0.1        no

/example                hello-foxx     Frank Celler         This is 'Hello World' for ArangoDB Foxx.          1.5.0      no

/hello                  hello-foxx     Frank Celler         This is 'Hello World' for ArangoDB Foxx.          1.5.0      no

---------------------   ------------   ------------------   -----------------------------------------------   --------   ------------

7 application(s) found

The current version of the application is 1.5.0 (check the output of list). It is even possible to mount a different version of an application.

Now let's remove the instance mounted under /hello.

unix> foxx-manager uninstall /hello

Application hello-foxx version 1.5.0 uninstalled successfully from mount point /hello

Note that "uninstall" will trigger the teardown script, which allows the application to clean up its own data. Afterwards the application will be deleted from disk.

We can also replace a running application by any other application:

unix> foxx-manager replace itzpapalotl:0.9.0 /example

Application itzpapalotl version 0.9.0 installed successfully at mount point /example

This is a shortcut for an uninstall then install procedure and includes invocation of teardown and setup scripts of the respective applications. If no application is installed at the mount point this shortcut will fail, use install instead. Note here we have specified a specific version of the application: 0.9.0.

Finally you can upgrade a running application to a newer version:

unix> foxx-manager update itzpapalotl:1.0.0 /example

Application itzpapalotl version 1.0.0 installed successfully at mount point /example

This will do uninstall and install of the application but will not execute setup or teardown scripts, but will run the upgrade script instead.

Application identifier

For all functions that install an application the first argument is an Application identifier. In the examples above we have used the Arango Store and downloaded applications that have been published there. But in most cases you will install you own application that is probably not published there.

The Application identifier supports several input formats:

- appname:version  Install an App from the ArangoDB store Read More

- git:user/repository:tag  Install an App from Github Read More

- http(s)://example.com/app.zip  Install an App from an URL Read More

- /usr/tmp/app.zip  Install an App from local file system Read More

- EMPTY  Generate a new Application Read More

Install Applications from Github

In this chapter we will make use of the Foxx manager as described before. This time we want to install an app out of our version control hosted on github.com.

In order to install an application we need three informations:

- Repository: The name of the repository.

- Username: The username of the user owning the repository.

- Version: (optional) branch or tag available on the repository.

As an example, we would like to install www.github.com/arangodb/hello-foxx. The username is arangodb, the repository is hello-foxx. If we do not define a version it will automatically install the master branch.

unix> foxx-manager install git:arangodb/hello-foxx /example

Application hello-foxx version 1.5.0 installed successfully at mount point /example

The hello-foxx app has defined a tag for version 1.4.4 that is named "v1.4.4". We can simply append this tag in the install command:

unix> foxx-manager install git:arangodb/hello-foxx:v1.4.4 /legacy

Application hello-foxx version 1.4.4 installed successfully at mount point /legacy

This reference for github repositories can be used in all functions of the Foxx-manager that allow to install Foxx applications:

install

unix> foxx-manager install git:arangodb/hello-foxx:v1.4.4 /legacy

Application hello-foxx version 1.4.4 installed successfully at mount point /legacy

replace

unix> foxx-manager replace git:arangodb/hello-foxx:v1.4.4 /legacy

Application hello-foxx version 1.4.4 installed successfully at mount point /legacy

upgrade

unix> foxx-manager upgrade git:arangodb/hello-foxx:v1.5.0 /legacy

Application hello-foxx version 1.5.0 installed successfully at mount point /legacy

Install Applications from Github

In this chapter we will make use of the Foxx manager as described before. This time we want to install an app out of our version control hosted on github.com. That means the Foxx-manager does not know about the application information and we cannot install it simply by its name.

In order to install an application we need three informations:

- Repository: The name of the repository.

- Username: The username of the user owning the repository.

- Version: Branch or Tag name available on the repository.

As an example, we would like to install https://www.github.com/arangodb/hello-foxx. The username is arangodb, the repository is hello-foxx. If we do not define a version it will automatically install the master branch.

unix> foxx-manager install git:arangodb/hello-foxx /example

Application hello-foxx version 1.5.0 installed successfully at mount point /example

The hello-foxx app has defined a tag for version 1.4.4 that is named "v1.4.4". We can simply append this tag in the install command:

unix> foxx-manager install git:arangodb/hello-foxx:v1.4.4 /legacy

Application hello-foxx version 1.4.4 installed successfully at mount point /legacy

This reference for github repositories can be used in all functions of the Foxx-manager that allow to install Foxx applications:

- install

unix> foxx-manager install git:arangodb/hello-foxx:v1.4.4 /legacy

Application hello-foxx version 1.4.4 installed successfully at mount point /legacy

- replace

unix> foxx-manager replace git:arangodb/hello-foxx:v1.4.4 /legacy

Application hello-foxx version 1.4.4 installed successfully at mount point /legacy

- upgrade

unix> foxx-manager upgrade git:arangodb/hello-foxx:v1.5.0 /legacy

Application hello-foxx version 1.5.0 installed successfully at mount point /legacy

Install Applications from local file system

In this chapter we will make use of the Foxx manager as described before. This time we want to install an app that is located on our local file system. At this point we have to mention that it if you connect to a remote ArangoDB with

unix> foxx-manager --server.endpoint tcp://example.com:8529

The file has to be available on your local machine, not on the remote server. The only thing you need is the path to your application either relative or absolute. You can install a Foxx application right from a directory:

unix> ls /Users/arangodb/hello-foxx

README.md     app.js        assets        files         kaffee.coffee lib

manifest.json models        scripts       thumbnail.png

unix> foxx-manager install /Users/arangodb/hello-foxx /example

Application hello-foxx version 1.5.0 installed successfully at mount point /example

Or you can pack the application into a zip archive. And then install using this archive.

unix> unzip -l ../hello-foxx.zip

Archive:  hello-foxx.zip

0836dc2e81be8264e480a7695b46c1abe7ef153d

Length     Date   Time    Name

--------    ----   ----    ----

0  09-10-14 15:35   hello-foxx/

1256  09-10-14 15:35   hello-foxx/README.md

11200  09-10-14 15:35   hello-foxx/app.js

0  09-10-14 15:35   hello-foxx/assets/

0  09-10-14 15:35   hello-foxx/assets/css/

82  09-10-14 15:35   hello-foxx/assets/css/base.css

86  09-10-14 15:35   hello-foxx/assets/css/custom.css

0  09-10-14 15:35   hello-foxx/assets/vendor/

0  09-10-14 15:35   hello-foxx/assets/vendor/bootstrap/

0  09-10-14 15:35   hello-foxx/assets/vendor/bootstrap/css/

22111  09-10-14 15:35   hello-foxx/assets/vendor/bootstrap/css/bootstrap-responsive.css

16849  09-10-14 15:35   hello-foxx/assets/vendor/bootstrap/css/bootstrap-responsive.min.css

127247  09-10-14 15:35   hello-foxx/assets/vendor/bootstrap/css/bootstrap.css

105939  09-10-14 15:35   hello-foxx/assets/vendor/bootstrap/css/bootstrap.min.css

0  09-10-14 15:35   hello-foxx/assets/vendor/bootstrap/img/

8777  09-10-14 15:35   hello-foxx/assets/vendor/bootstrap/img/glyphicons-halflings-white.png

12799  09-10-14 15:35   hello-foxx/assets/vendor/bootstrap/img/glyphicons-halflings.png

0  09-10-14 15:35   hello-foxx/assets/vendor/jquery/

268380  09-10-14 15:35   hello-foxx/assets/vendor/jquery/jquery.js

0  09-10-14 15:35   hello-foxx/assets/vendor/sh/

1981  09-10-14 15:35   hello-foxx/assets/vendor/sh/highlighter.css

5563  09-10-14 15:35   hello-foxx/assets/vendor/sh/sh_javascript.js

5305  09-10-14 15:35   hello-foxx/assets/vendor/sh/sh_main.min.js

0  09-10-14 15:35   hello-foxx/files/

3266  09-10-14 15:35   hello-foxx/files/index.html

398  09-10-14 15:35   hello-foxx/files/static.html

361  09-10-14 15:35   hello-foxx/kaffee.coffee

0  09-10-14 15:35   hello-foxx/lib/

108  09-10-14 15:35   hello-foxx/lib/a.js

43  09-10-14 15:35   hello-foxx/lib/c.js

1129  09-10-14 15:35   hello-foxx/manifest.json

0  09-10-14 15:35   hello-foxx/models/

330  09-10-14 15:35   hello-foxx/models/tiger.js

0  09-10-14 15:35   hello-foxx/scripts/

2065  09-10-14 15:35   hello-foxx/scripts/setup.js

1798  09-10-14 15:35   hello-foxx/scripts/teardown.js

17727  09-10-14 15:35   hello-foxx/thumbnail.png

--------                   -------

614800                   37 files

unix> foxx-manager install ../hello-foxx.zip /example

Application hello-foxx version 1.5.0 installed successfully at mount point /example

You can use paths to directories in all functions of the Foxx-manager that allow to install Foxx applications:

install

unix> foxx-manager install /Users/arangodb/hello-foxx /example

Application hello-foxx version 1.5.0 installed successfully at mount point /example

replace

unix> foxx-manager replace /Users/arangodb/hello-foxx /example

Application hello-foxx version 1.5.0 installed successfully at mount point /example

upgrade

unix> foxx-manager upgrade /Users/arangodb/hello-foxx /example

Application hello-foxx version 1.5.0 installed successfully at mount point /example

Install Applications from remote host

In this chapter we will make use of the Foxx manager as described before. This time we want to install an app hosted on a server. Currently the Foxx-manager supports downloads of applications via HTTP and HTTPS protocols.

Remote file format

The file on the remote server has to be a valid Foxx application packed in a zip archive. The zip archive has to contain a top-level directory which is containing the sources of the Foxx application. Especially the file manifest.json has to be directly contained in this top-level directory. On an Unix system you can verify that your zip archive has the correct format if the content looks like this:

unix> unzip -l hello-foxx.zip

Archive:  hello-foxx.zip

0836dc2e81be8264e480a7695b46c1abe7ef153d

Length     Date   Time    Name

--------    ----   ----    ----

0  09-10-14 15:35   hello-foxx/

1256  09-10-14 15:35   hello-foxx/README.md

11200  09-10-14 15:35   hello-foxx/app.js

0  09-10-14 15:35   hello-foxx/assets/

0  09-10-14 15:35   hello-foxx/assets/css/

82  09-10-14 15:35   hello-foxx/assets/css/base.css

86  09-10-14 15:35   hello-foxx/assets/css/custom.css

0  09-10-14 15:35   hello-foxx/assets/vendor/

0  09-10-14 15:35   hello-foxx/assets/vendor/bootstrap/

0  09-10-14 15:35   hello-foxx/assets/vendor/bootstrap/css/

22111  09-10-14 15:35   hello-foxx/assets/vendor/bootstrap/css/bootstrap-responsive.css

16849  09-10-14 15:35   hello-foxx/assets/vendor/bootstrap/css/bootstrap-responsive.min.css

127247  09-10-14 15:35   hello-foxx/assets/vendor/bootstrap/css/bootstrap.css

105939  09-10-14 15:35   hello-foxx/assets/vendor/bootstrap/css/bootstrap.min.css

0  09-10-14 15:35   hello-foxx/assets/vendor/bootstrap/img/

8777  09-10-14 15:35   hello-foxx/assets/vendor/bootstrap/img/glyphicons-halflings-white.png

12799  09-10-14 15:35   hello-foxx/assets/vendor/bootstrap/img/glyphicons-halflings.png

0  09-10-14 15:35   hello-foxx/assets/vendor/jquery/

268380  09-10-14 15:35   hello-foxx/assets/vendor/jquery/jquery.js

0  09-10-14 15:35   hello-foxx/assets/vendor/sh/

1981  09-10-14 15:35   hello-foxx/assets/vendor/sh/highlighter.css

5563  09-10-14 15:35   hello-foxx/assets/vendor/sh/sh_javascript.js

5305  09-10-14 15:35   hello-foxx/assets/vendor/sh/sh_main.min.js

0  09-10-14 15:35   hello-foxx/files/

3266  09-10-14 15:35   hello-foxx/files/index.html

398  09-10-14 15:35   hello-foxx/files/static.html

361  09-10-14 15:35   hello-foxx/kaffee.coffee

0  09-10-14 15:35   hello-foxx/lib/

108  09-10-14 15:35   hello-foxx/lib/a.js

43  09-10-14 15:35   hello-foxx/lib/c.js

1129  09-10-14 15:35   hello-foxx/manifest.json

0  09-10-14 15:35   hello-foxx/models/

330  09-10-14 15:35   hello-foxx/models/tiger.js

0  09-10-14 15:35   hello-foxx/scripts/

2065  09-10-14 15:35   hello-foxx/scripts/setup.js

1798  09-10-14 15:35   hello-foxx/scripts/teardown.js

17727  09-10-14 15:35   hello-foxx/thumbnail.png

--------                   -------

614800                   37 files

Next you have to make this file publicly available over HTTP or HTTPS on a webserver. Assume we can download the app at http://www.example.com/hello.zip.

Install from remote server

Having the link to your Foxx application at hand you can just hand this link over to the Foxx manager:

unix> foxx-manager install http://www.example.com/hello.zip /example

Application hello-foxx version 1.5.0 installed successfully at mount point /example

ArangoDB will try to download and extract the file stored at the remote location.

This HTTP or HTTPS link can be used in all functions of the Foxx-manager that allow to install Foxx applications:

install

unix> foxx-manager install http://www.example.com/hello.zip /example

Application hello-foxx version 1.5.0 installed successfully at mount point /example

replace

unix> foxx-manager replace http://www.example.com/hello.zip /example

Application hello-foxx version 1.5.0 installed successfully at mount point /example

upgrade

unix> foxx-manager upgrade http://www.example.com/hello.zip /example

Application hello-foxx version 1.5.0 installed successfully at mount point /example

Generate a new Application

In this chapter we will make use of the Foxx manager as described before. This time we do not have anything to start from and would like ArangoDB to create a template for us. For a generated application you only need to be prepared to answer some questions for meta information, don't worry if you do not yet have an answer, all of this info is optional and can be inserted later:

- Your name (e.g.  Author )

- The name of your application (e.g.  MyApp )

- A description what your application should do (e.g.  An application to access ArangoDB )

- The license you want your application to have (e.g.  Apache 2 )

- Names of collections you only want this application to access (e.g.  ["MyCollection"] )

Now you can create a new empty application with just one line:

unix> foxx-manager install EMPTY /example '{"author": "Author", "name": "MyApp", "description": "An application to access ArangoDB", "license": "Apache 2", "collectionNames": ["MyCollection"]}'

Application MyApp version 0.0.1 installed successfully at mount point /example

options used: {"author":"Author","name":"MyApp","description":"An application to access ArangoDB","license":"Apache 2","collectionNames":["MyCollection"],"configuration":{} }

This action now has generated you a fresh application in version 0.0.1. It has set up all meta-information you have provided For each collection in  collectionNames  it has created a collection and generated all 5 CRUD routes: List all, Create one, Read one, Update one, Delete one.

You can use the generator in all functions of the Foxx-manager that allows to install Foxx applications:

install

unix> foxx-manager install EMPTY /example '{"author": "Author", "name": "MyApp", "description": "An application to access ArangoDB", "license": "Apache 2", "collectionNames": ["MyCollection"]}'

Application MyApp version 0.0.1 installed successfully at mount point /example

options used: {"author":"Author","name":"MyApp","description":"An application to access ArangoDB","license":"Apache 2","collectionNames":["MyCollection"],"configuration":{} }

replace

unix> foxx-manager replace EMPTY /example '{"author": "Author", "name": "MyApp", "description": "An application to access ArangoDB", "license": "Apache 2", "collectionNames": ["MyCollection"]}'

Application MyApp version 0.0.1 installed successfully at mount point /example

options used: {"author":"Author","name":"MyApp","description":"An application to access ArangoDB","license":"Apache 2","collectionNames":["MyCollection"],"configuration":{} }

upgrade

unix> foxx-manager upgrade EMPTY /example '{"author": "Author", "name": "MyApp", "description": "An application to access ArangoDB", "license": "Apache 2", "collectionNames": ["MyCollection"]}'

Application MyApp version 0.0.1 installed successfully at mount point /example

options used: {"author":"Author","name":"MyApp","description":"An application to access ArangoDB","license":"Apache 2","collectionNames":["MyCollection"],"configuration":{} }

Develop your own Foxx

This chapter will explain to you how to write a Foxx on your own and use it to enhance the ArangoDB's functionality. Be it a microservice, an API with a user interface or an internal library.

Before reading this chapter you should make sure to at least read one of the install sections beforehand to get a good staring point. Recommended is the generate section to get started with your own Foxx, but you can also start with an existing one.

Development Mode

At first we will introduce the development mode and describe its side effects. You can skip this section if you do not have access to the file system of ArangoDB as you will not get the benefits of this mode. You will have to stick to the procedure described in New Versions.

Read More

Debugging

Next you will learn about the debugging mechanisms if you have set a Foxx into development mode. The app will return more debugging information in this mode like stacktraces. In production mode stacktraces will be kept internally.

Read More

Folder structure

If you want to get started with coding this is the section to begin with. It will introduce the folder structure and describes where which files have to be located on server side.

Read More

Framework and tools

Now we are entering the reference documentation of tools available in the Foxx framework. The tools contain:

- Controller

- Setup & Teardown

- Repository

- Model

- Queries

- Background Tasks

- Console API

- you may use Javascript Modules and install node modules using npm in the directory containing your  manifest.json .

Finally we want to apply some meta information to the Foxx. How this is done is described in the Meta information chapter.

Application Development Mode

This chapter describes the development mode for Foxx applications. It is only useful if you have write access to the files in ArangoDB's application folder. The folder is printed in the web interface for development Foxxes and can be reconfigured using the startup option  --javascript.app-path . If you do not have access to this folder, e.g. using a hosted service like myArangoDB, you cannot make use of this feature. You will have to stick to the procedure described in New Versions.

Before 2.5 the startup option `--javascript.dev-app-path` was required for the development mode. This caused a lot of confusion and introduced problems when moving from development to production. So we decided to unify both app paths and activate development mode for specific Apps during runtime. The `--javascript.dev-app-path` parameter is not having any effect any more.

Activation

Activating the development mode is done with a single command:

unix> foxx-manager development /example

Activated development mode for Application hello-foxx version 1.5.0 on mount point /example

Now the app will now be listed in listDevelopment:

unix> foxx-manager listDevelopment

Mount       Name          Author          Description                                 Version    Development

---------   -----------   -------------   -----------------------------------------   --------   ------------

/example    hello-foxx    Frank Celler    This is 'Hello World' for ArangoDB Foxx.    1.5.0      true

---------   -----------   -------------   -----------------------------------------   --------   ------------

1 application(s) found

Effects

For a Foxx application in development mode the following effects apply:

Reload on request Whenever a request is routed to this application its source is reloaded. This means all requests are slightly slower than in production mode. But you will get immediate live updates on your code changes.

Exposed Debugging information This application will deliver error messages and stack traces to requesting client. For more information see the Debugging section.

Available Debugging mechanisms

We are talking about the development mode for Foxx. Hence one of the most important parts will be debugging of your Foxx. We have several mechanisms available to simplify this task. During Foxx development we assume the following situation: You have installed a syntactically valid version of a Foxx and want to develop it further. You have activated the development mode for this route:

unix>foxx-manager development /example

Activated development mode for Application hello-foxx version 1.5.0 on mount point /example

Errors during install

Now you apply changes to the source code of Foxx. In development mode it is possible that you create a Foxx that could not be installed regularly. If this is the case and you request any route of it you should receive a detailed error information:

unix>curl -X GET --dump - http://localhost:8529/_db/_system/example/failed

HTTP/1.1 500 Internal Error

Server: ArangoDB

Connection: Keep-Alive

Content-Type: application/json; charset=utf-8

Content-Length: 554

{"exception":"Error: App not found","stacktrace":["Error: App not found","  at Object.lookupApp (./js/server/modules/org/arangodb/foxx/manager.js:99:13)","  at foxxRouting (./js/server/modules/org/arangodb/actions.js:1040:27)","  at execute (./js/server/modules/org/arangodb/actions.js:1291:7)","  at Object.routeRequest (./js/server/modules/org/arangodb/actions.js:1312:3)","  at Function.actions.defineHttp.callback (js/actions/api-system.js:51:15)",""],"error":true,"code":500,"errorNum":500,"errorMessage":"failed to load foxx mounted at '/example'"}

Errors in routes

If you have created a Foxx that can be regularly installed but has an unhandled error inside a route. Triggering this route and entering the error case will return the specific error including a stack trace for you to hunt it down:

unix>curl -X GET http://localhost:8529/_db/_system/example/failed

HTTP/1.1 500 Internal Error

Server: ArangoDB

Connection: Keep-Alive

Content-Type: application/json

Content-Length: 917

{"error":"Unhandled Error","stack":"Error: Unhandled Error\n  at fail (js/apps/_db/_system/example/APP/app.js:279:13)\n  at js/apps/_db/_system/example/APP/app.js:284:5\n  at Object.res.action.callback (./js/server/modules/org/arangodb/foxx/internals.js:108:5)\n  at ./js/server/modules/org/arangodb/foxx/routing.js:346:19\n  at execute (./js/server/modules/org/arangodb/actions.js:1291:7)\n  at next (./js/server/modules/org/arangodb/actions.js:1308:7)\n  at [object Object]:386:5\n  at execute (./js/server/modules/org/arangodb/actions.js:1291:7)\n  at routeRequest (./js/server/modules/org/arangodb/actions.js:1312:3)\n  at foxxRouting (./js/server/modules/org/arangodb/actions.js:1082:7)\n  at execute (./js/server/modules/org/arangodb/actions.js:1291:7)\n  at Object.routeRequest (./js/server/modules/org/arangodb/actions.js:1312:3)\n  at Function.actions.defineHttp.callback (js/actions/api-system.js:51:15)\n"}

Errors in logs

Independent of the errors presented in the routes on requests Fox will always log errors to the log-file if caught by the default error handlers. The log entries will always contain stacktraces and error messages:

INFO /example, incoming request from 127.0.0.1: GET http://0.0.0.0:8529/example/failed

ERROR Error in foxx route '{ "match" : "/failed", "methods" : [ "get" ] }': 'Unhandled Error', Stacktrace: Error: Unhandled Error

ERROR   at fail (js/apps/_db/_system/example/APP/app.js:279:13)

ERROR   at js/apps/_db/_system/example/APP/app.js:284:5

ERROR   at Object.res.action.callback (./js/server/modules/org/arangodb/foxx/internals.js:108:5)

ERROR   at ./js/server/modules/org/arangodb/foxx/routing.js:346:19

ERROR   at execute (./js/server/modules/org/arangodb/actions.js:1291:7)

ERROR   at next (./js/server/modules/org/arangodb/actions.js:1308:7)

ERROR   at [object Object]:386:5

ERROR   at execute (./js/server/modules/org/arangodb/actions.js:1291:7)

ERROR   at routeRequest (./js/server/modules/org/arangodb/actions.js:1312:3)

ERROR   at foxxRouting (./js/server/modules/org/arangodb/actions.js:1082:7)

ERROR   at execute (./js/server/modules/org/arangodb/actions.js:1291:7)

ERROR   at Object.routeRequest (./js/server/modules/org/arangodb/actions.js:1312:3)

ERROR   at Function.actions.defineHttp.callback (js/actions/api-system.js:51:15)

ERROR

INFO /example, outgoing response with status 500 of type application/json, body length: 917

Folder Structure

Now we are almost ready to write some code. Hence it is time to introduce the folder structure created by Foxx. We still follow the example of the app installed at  /example . The route to reach this application via HTTP(S) is constructed with the following parts:

- The ArangoDB endpoint  <arangodb> : (e.g.  http://localhost:8529 )

- The selected database  <db> : (e.g.  _system )

- The mount point  <mount> : (e.g.  /example )

Now the route is constructed as follows:

<arangodb>/_db/<db>/<mount>

http://localhost:8529/_db/_system/example

For the sources of the application the path on your file system the path is constructed almost the same way. But here we need some additional information:

- The app-path  <app-path> : (e.g.  /var/lib/arangodb-apps )

- The selected database  <db> : (e.g.  _system )

- The mount point  <mount> : (e.g.  /example )

Note: You can set your app-path to an arbitrary folder using the  --javascript.app-path  startup parameter. Now the path is constructed as follows:

<app-path>/_db/<db>/<mount>/APP

Linux: /var/lib/arangodb-apps/_db/_system/example/APP

Mac: /usr/local/var/lib/arangodb-apps/_db/_system/example/APP

Windows: C:\Program Files\ArangoDB\js\apps\_db\_system\example\APP

Before 2.5 the folder was constructed using application name and version. That was necessary because installation was a two step process:

1. Including the Application sources into ArangoDB (and creating the folder)

2. Mounting the application to one specific mountpoint

This caused some confusion and a lot of unnecessary administration overhead. One had to remember which apps are just known to ArangoDB, which ones are actually executed in which version etc. The use-case we actually introduced this staging for was heavy reuse of equal apps. However it turned out that this is rarely the case and the overhead by having redundant sources is small compared to the improved user experience not having this staging. So we decided to entirely remove the staging and make installation an one step process without caching old versions of an app. This means if you now uninstall an application it is removed from file system. Before 2.5 you had to purge the application to make sure it is removed.

Now you can start modifying the files located there. As a good entry you should start with the Controller

Details on Controller

Create

new Controller(applicationContext, options)

This creates a new Controller. The first argument is the controller context available in the variable applicationContext. The second one is an options array with the following attributes:

- urlPrefix: All routes you define within will be prefixed with it.

Examples

app = new Controller(applicationContext, {

urlPrefix: "/meadow"

});

HTTP Methods

get

Controller.get(path, callback)

Defines a new route on  path  that handles requests from the HTTP verb  get . This route can also be 'parameterized' like  /goose/:barn . In this case you can later get the value the user provided for  barn  via the  params  function in the  request . The function defined in  callback  will be invoked whenever this type of request is recieved.  callback  get's two arguments  request  and  response , see below for further information about these objects.

Examples

app.get('/goose/barn', function (req, res) {

// Take this request and deal with it!

});

head

Controller.head(path, callback)

Defines a new route on  path  that handles requests from the HTTP verb  head . This route can also be 'parameterized' like  /goose/:barn . In this case you can later get the value the user provided for  barn  via the  params  function in the  request . The function defined in  callback  will be invoked whenever this type of request is recieved.  callback  get's two arguments  request  and  response , see below for further information about these objects.

Examples

app.head('/goose/barn', function (req, res) {

// Take this request and deal with it!

});

post

Controller.post(path, callback)

Defines a new route on  path  that handles requests from the HTTP verb  post . This route can also be 'parameterized' like  /goose/:barn . In this case you can later get the value the user provided for  barn  via the  params  function in the  request . The function defined in  callback  will be invoked whenever this type of request is recieved.  callback  get's two arguments  request  and  response , see below for further information about these objects.

Examples

app.post('/goose/barn', function (req, res) {

// Take this request and deal with it!

});

put

Controller.put(path, callback)

Defines a new route on  path  that handles requests from the HTTP verb  put . This route can also be 'parameterized' like  /goose/:barn . In this case you can later get the value the user provided for  barn  via the  params  function in the  request . The function defined in  callback  will be invoked whenever this type of request is recieved.  callback  get's two arguments  request  and  response , see below for further information about these objects.

Examples

app.put('/goose/barn', function (req, res) {

// Take this request and deal with it!

});

patch

Controller.patch(path, callback)

Defines a new route on  path  that handles requests from the HTTP verb  patch . This route can also be 'parameterized' like  /goose/:barn . In this case you can later get the value the user provided for  barn  via the  params  function in the  request . The function defined in  callback  will be invoked whenever this type of request is recieved.  callback  get's two arguments  request  and  response , see below for further information about these objects.

Examples

app.patch('/goose/barn', function (req, res) {

// Take this request and deal with it!

});

delete

Controller.delete(path, callback)

Defines a new route on  path  that handles requests from the HTTP verb  delete . This route can also be 'parameterized' like  /goose/:barn . In this case you can later get the value the user provided for  barn  via the  params  function in the  request . The function defined in  callback  will be invoked whenever this type of request is recieved.  callback  get's two arguments  request  and  response , see below for further information about these objects.

Examples

app.delete('/goose/barn', function (req, res) {

// Take this request and deal with it!

});

Documenting and constraining a specific route

If you now want to document your route, you can use JSDoc style comments (a multi-line comment block where the first line starts with /** instead of /*) above your routes to do that:

/** Get all foxxes

*

* If you want to get all foxxes, please use this

* method to do that.

*/

app.get("/foxxes", function () {

// ...

});

The first line will be treated as a summary (For optical reasons in the produced documentation, the summary is restricted to 60 characters). All following lines will be treated as additional notes shown in the detailed view of the route documentation. With the provided information, Foxx will generate a nice documentation for you. Furthermore you can describe your API by chaining the following methods onto your path definition:

pathParam

Route.pathParam(id, options)

If you defined a route "/foxx/:name", containing a parameter called  name  you can constrain which format this parameter is allowed to have. This format is defined using joi in the  options  parameter. Using this function will at first allow you to access this parameter in your route handler using  req.params(id) , will reject any request having a paramter that does not match the joi definition and creates a documentation for this parameter in ArangoDBs WebInterface.

For more information on joi see the official Joi documentation.

Parameter

- id: name of the param.

- options: a joi schema or an object with the following properties:

- type: a joi schema.

- description: documentation description for the parameter.

- required (optional): whether the parameter is required. Default: determined by type.

Examples

app.get("/foxx/:name", function {

// Do something

}).pathParam("name", joi.string().required().description("Name of the Foxx"));

You can also pass in a configuration object instead:

app.get("/foxx/:name", function {

// Do something

}).pathParam("name", {

type: joi.string(),

required: true,

description: "Name of the Foxx"

});

queryParam

Route.queryParam(id, options)

Describe a query parameter:

If you defined a route "/foxx", you can allow a query paramter with the name  id  on it and constrain the format of this parameter by giving it a joi type in the  options  parameter. Using this function will at first allow you to access this parameter in your route handler using  req.params(id) , will reject any request having a paramter that does not match the joi definition and creates a documentation for this parameter in ArangoDBs WebInterface.

For more information on joi see the official Joi documentation.

You can also provide a description of this parameter and whether you can provide the parameter multiple times.

Parameter

- id: name of the parameter

- options: a joi schema or an object with the following properties:

- type: a joi schema

- description: documentation description for this param.

- required (optional): whether the param is required. Default: determined by type.

- allowMultiple (optional): whether the param can be specified more than once. Default:  false .

Examples

app.get("/foxx", function {

// Do something

}).queryParam("id",

joi.string()

.required()

.description("Id of the Foxx")

.meta({allowMultiple: false})

});

You can also pass in a configuration object instead:

app.get("/foxx", function {

// Do something

}).queryParam("id", {

type: joi.string().required().description("Id of the Foxx"),

allowMultiple: false

});

bodyParam

Route.bodyParam(paramName, options)

Defines that this route expects a JSON body when requested and binds it to a pseudo parameter with the name  paramName . The body can than be read in the the handler using  req.params(paramName)  on the request object. In the  options  parameter you can define how a valid request body should look like. This definition can be done in two ways, either using joi directly. Accessing the body in this case will give you a JSON object. The other way is to use a Foxx Model. Accessing the body in this case will give you an instance of this Model. For both ways an entry for the body will be added in the Documentation in ArangoDBs WebInterface. For information about how to annotate your models, see the Model section. All requests sending a body that does not match the validation given this way will automatically be rejected.

You can also wrap the definition into an array, in this case this route expects a body of type array containing arbitrary many valid objects. Accessing the body parameter will then of course return an array of objects.

Note: The behavior of  bodyParam  changes depending on the  rootElement  option set in the manifest. If it is set to  true , it is expected that the body is an object with a key of the same name as the  paramName  argument. The value of this object is either a single object or in the case of a multi element an array of objects.

Parameter

- paramName: name of the body parameter in  req.parameters .

- options: a joi schema or an object with the following properties:

- description: the documentation description of the request body.

- type: the Foxx model or joi schema to use.

- allowInvalid (optional):  true  if validation should be skipped. (Default:  false )

Examples

app.post("/foxx", function (req, res) {

var foxxBody = req.parameters.foxxBody;

// Do something with foxxBody

}).bodyParam("foxxBody", {

description: "Body of the Foxx",

type: FoxxBodyModel

});

Using a joi schema:

app.post("/foxx", function (req, res) {

var joiBody = req.parameters.joiBody;

// Do something with the number

}).bodyParam("joiBody", {

type: joi.number().integer().min(5),

description: "A number greater than five",

allowInvalid: false // default

});

Shorthand version:

app.post("/foxx", function (req, res) {

var joiBody = req.parameters.joiBody;

// Do something with the number

}).bodyParam(

"joiBody",

joi.number().integer().min(5)

.description("A number greater than five")

.meta({allowInvalid: false}) // default

);

errorResponse

Route.errorResponse(errorClassOrName, code, description, [callback])

Define a reaction to a thrown error for this route: If your handler throws an error of the errorClass defined in  errorClassOrName  or the error has an attribute  name  equal to  errorClassOrName , it will be caught and the response object will be filled with the given status code and a JSON with error set to your description as the body.

If you want more control over the returned JSON, you can give an optional fourth parameter in form of a function. It gets the error as an argument, the return value will be transformed into JSON and then be used as the body. The status code will be used as described above. The description will be used for the documentation.

It also adds documentation for this error response to the generated documentation.

Examples

/* define our own error type, FoxxyError */

var FoxxyError = function (message) {

this.name = "FError";

this.message = "the following FoxxyError occurred: " + message;

};

FoxxyError.prototype = new Error();

app.get("/foxx", function {

/* throws a FoxxyError */

throw new FoxxyError();

}).errorResponse(FoxxyError, 303, "This went completely wrong. Sorry!");

app.get("/foxx", function {

throw new FoxxyError("oops!");

}).errorResponse("FError", 303, "This went completely wrong. Sorry!", function (e) {

return {

code: 123,

desc: e.message

};

});

onlyif

Route.onlyIf(check)

This functionality is used to secure a route by applying a checking function on the request beforehand, for example the check authorization. It expects  check  to be a function that takes the request object as first parameter. This function is executed before the actual handler is invoked. If  check  throws an error the actual handler will not be invoked. Remember to provide an  errorResponse  on the route as well to define the behavior in this case.

Examples

app.get("/foxx", function {

// Do something

}).onlyIf(aFunction).errorResponse(ErrorClass, 303, "This went completely wrong. Sorry!");

onlyIfAuthenticated

FoxxController#onlyIfAuthenticated(code, reason)

Please activate sessions for this app if you want to use this function. Or activate authentication (deprecated). If the user is logged in, it will do nothing. Otherwise it will respond with the status code and the reason you provided (the route handler won't be called). This will also add the according documentation for this route.

Examples

app.get("/foxx", function {

// Do something

}).onlyIfAuthenticated(401, "You need to be authenticated");

summary

Route.summary(description)

Set the summary for this route in the documentation. Can't be longer than 8192 characters. This is equal to using JavaDoc style comments right above your function. If you provide both comment and  summary()  the call to  summary()  wins and will be used.

Examples

Version with comment:

/** Short description

*

* Longer description

* with multiple lines

*/

app.get("/foxx", function() {

});

is identical to:

app.get("/foxx", function() {

})

.summary("Short description")

.notes(["Longer description", "with multiple lines"]);

notes

Route.notes(...description)

Set the long description for this route in the documentation // Examples

Version with comment:

/** Short description

*

* Longer description

* with multiple lines

*/

app.get("/foxx", function() {

});

is identical to:

app.get("/foxx", function() {

})

.summary("Short description")

.notes(["Longer description", "with multiple lines"]);

extend

In many use-cases several of the functions are always used in a certain combination (e.g.:  onlyIf  with  errorResponse ). In order to avoid duplicating this equal usage for several routes in your application you can extend the controller with your own functions. These functions can simply combine several of the above on a single name, so you only have to invoke your self defined single function on all routes using these extensions.

Controller.extend(extensions)

Extends all functions to define routes in this controller. This allows to combine several route extensions with the invocation of a single function. This is especially useful if you use the same input parameter in several routes of your controller and want to apply the same validation, documentation and error handling for it.

The  extensions  parameter is a JSON object with arbitrary keys. Each key is used as the name of the function you want to define (you cannot overwrite internal functions like  pathParam ) and the value is a function that will be invoked. This function can get arbitrary many arguments and the  this  of the function is bound to the route definition object (e.g. you can use  this.pathParam() ). Your newly defined function is chainable similar to the internal functions.

Examples

Define a validator for a queryParameter, including documentation and errorResponses in a single command:

controller.extend({

myParam: function (maxValue) {

this.queryParam("value", {type: joi.number().required()});

this.onlyIf(function(req) {

var v = req.param("value");

if (v > maxValue) {

throw new NumberTooLargeError();

}

});

this.errorResponse(NumberTooLargeError, 400, "The given value is too large");

}

});

controller.get("/goose/barn", function(req, res) {

// Will only be invoked if the request has parameter value and it is less or equal 5.

}).myParam(5);

Documenting and constraining all routes

In addition to documenting a specific route, you can also do the same for all routes of a controller. For this purpose use the allRoutes object of the according controller. The following methods are available.

Examples

Provide an error response for all routes handled by this controller:

ctrl.allRoutes

.errorResponse(Unauthorized, 401, 'Not authenticated.')

.errorResponse(NotFound, 404, 'Document not found.')

.errorResponse(ImATeapot, 418, 'I\'m a teapot.');

ctrl.get('/some/route', function (req, res) {

// ...

throw new NotFound('The document does not exist');

// ...

}); // no errorResponse needed here

ctrl.get('/another/route', function (req, res) {

// ...

throw new NotFound('I made you a cookie but I ated it');

// ...

}); // no errorResponse needed here either

errorResponse

Controller.allRoutes.errorResponse(errorClass, code, description)

This is equal to invoking  Route.errorResponse  on all routes bound to this controller.

Examples

app.allRoutes.errorResponse(FoxxyError, 303, "This went completely wrong. Sorry!");

app.get("/foxx", function {

// Do something

});

onlyIf

Controller.allRoutes.onlyIf(code, reason)

This is equal to invoking  Route.onlyIf  on all routes bound to this controller.

Examples

app.allRoutes.onlyIf(myPersonalCheck);

app.get("/foxx", function {

// Do something

});

onlyIfAuthenticated

Controller.allRoutes.onlyIfAuthenticated(code, description)

This is equal to invoking  Route.onlyIfAuthenticated  on all routes bound to this controller.

Examples

app.allRoutes.onlyIfAuthenticated(401, "You need to be authenticated");

app.get("/foxx", function {

// Do something

});

pathParam

Controller.allRoutes.pathParam(id, options)

This is equal to invoking  Route.pathParam  on all routes bound to this controller.

Examples

app.allRoutes.pathParam("id", joi.string().required().description("Id of the Foxx"));

app.get("/foxx/:id", function {

// Secured by pathParam

});

You can also pass in a configuration object instead:

app.allRoutes.pathParam("id", {

type: joi.string(),

required: true,

description: "Id of the Foxx"

});

app.get("/foxx/:id", function {

// Secured by pathParam

});

bodyParam

Controller.allRoutes.queryParam(id, options)

This is equal to invoking  Route.queryParam  on all routes bound to this controller.

Examples

app.allroutes.queryParam("id",

joi.string()

.required()

.description("Id of the Foxx")

.meta({allowMultiple: false})

});

app.get("/foxx", function {

// Do something

});

You can also pass in a configuration object instead:

app.allroutes.queryParam("id", {

type: joi.string().required().description("Id of the Foxx"),

allowMultiple: false

});

app.get("/foxx", function {

// Do something

});

Before and After Hooks

You can use the following two functions to do something before or respectively after the normal routing process is happening. You could use that for logging or to manipulate the request or response (translate it to a certain format for example).

before

Controller.before(path, callback)

Defines an additional function on the route  path  which will be executed before the callback defined for a specific HTTP verb is executed. The  callback  function has the same signature as the  callback  in the specific route. You can also omit the  path , in this case  callback  will be executed before handleing any request in this Controller.

If  callback  returns the Boolean value  false , the route handling will not proceed. You can use this to intercept invalid or unauthorized requests and prevent them from being passed to the matching routes.

Examples

app.before('/high/way', function(req, res) {

//Do some crazy request logging

});

after

Controller.after(path, callback)

Similar to  Controller.before(path, callback)  but  callback  will be invoked after the request is handled in the specific route.

Examples

app.after('/high/way', function(req, res) {

//Do some crazy response logging

});

around

Controller.around(path, callback)

Similar to  Controller.before(path, callback)   callback  will be invoked instead of the specific handler.  callback  takes two additional paramaters  opts  and  next  where  opts  contains options assigned to the route and  next  is a function. Whenever you call  next  in  callback  the specific handler is invoked, if you do not call  next  the specific handler will not be invoked at all. So using around you can execute code before and after a specific handler and even call the handler only under certain circumstances. If you omit  path   callback  will be called on every request.

Examples

app.around('/high/way', function(req, res, opts, next) {

//Do some crazy request logging

next();

//Do some more crazy request logging

});

The Request and Response Objects

When you have created your FoxxController you can now define routes on it. You provide each with a function that will handle the request. It gets two arguments (four, to be honest. But the other two are not relevant for now):

- The request object

- The response object

These objects are provided by the underlying ArangoDB actions and enhanced by the BaseMiddleware provided by Foxx.

The Request Object

The request object inherits several attributes from the underlying Actions:

- compatibility: an integer specifying the compatibility version sent by the client (in request header x-arango-version). If the client does not send this header, ArangoDB will set this to the minimum compatible version number. The value is 10000 major + 100 minor (e.g. 10400 for ArangoDB version 1.4).

- user: the name of the current ArangoDB user. This will be populated only if authentication is turned on, and will be null otherwise.

- database: the name of the current database (e.g. _system)

- protocol: http or https

- server: a JSON object with sub-attributes address (containing server host name or IP address) and port (server port).

- path: request URI path, with potential database name stripped off.

- url: request URI path + query string, with potential database name stripped off

- headers: a JSON object with the request headers as key/value pairs. Note: All header field names are lower-cased

- cookies: a JSON object with the request cookies as key/value pairs

- requestType: the request method (e.g. "GET", "POST", "PUT", ...)

- requestBody: the complete body of the request as a string

- parameters: a JSON object with all parameters set in the URL as key/value pairs

- urlParameters: a JSON object with all named parameters defined for the route as key/value pairs.

In addition to these attributes, a Foxx request objects provides the following convenience methods:

body

request.body()

Get the JSON parsed body of the request. If you need the raw version, please refer to the rawBody function.

rawBody

request.rawBody()

The raw request body, not parsed. The body is returned as a UTF-8 string. Note that this can only be used sensibly if the request body contains valid UTF-8. If the request body is known to contain non-UTF-8 data, the request body can be accessed by using  request.rawBodyBuffer .

rawBodyBuffer

request.rawBodyBuffer()

The raw request body, returned as a Buffer object.

params

request.params(key)

Get the parameters of the request. This process is two-fold:

- If you have defined an URL like /test/:id and the user requested /test/1, the call params("id") will return 1.

- If you have defined an URL like /test and the user gives a query component, the query parameters will also be returned. So for example if the user requested /test?a=2, the call params("a") will return 2.

cookie

request.cookie(name, cfg)

Read a cookie from the request. Optionally the cookie's signature can be verified.

Parameter

- name: the name of the cookie to read from the request.

- cfg (optional): an object with any of the following properties:

- signed (optional): an object with any of the following properties:

- secret: a secret string that was used to sign the cookie.

- algorithm: hashing algorithm that was used to sign the cookie. Default: "sha256".

If signed is a string, it will be used as the secret instead.

If a secret is provided, a second cookie with the name name + ".sig" will be read and its value will be verified as the cookie value's signature.

If the cookie is not set or its signature is invalid, "undefined" will be returned instead.

Examples

var sid = request.cookie("sid", {signed: "keyboardcat"});

requestParts

Only useful for multi-part requests.

request.requestParts()

Returns an array containing the individual parts of a multi-part request. Each part contains a  headers  attribute with all headers of the part, and a  data  attribute with the content of the part in a Buffer object. If the request is not a multi-part request, this function will throw an error.

The Response Object

Every response object has the body attribute from the underlying Actions to set the raw body by hand.

You provide your response body as a string here.

Response status

response.status(code)

Set the status code of your response, for example:

Examples

response.status(404);

Response set

response.set(key, value)

Set a header attribute, for example:

Examples

response.set("Content-Length", 123);

response.set("Content-Type", "text/plain");

or alternatively:

response.set({

"Content-Length": "123",

"Content-Type": "text/plain"

});

Response json

response.json(object)

Set the content type to JSON and the body to the JSON encoded object you provided.

Examples

response.json({'born': 'December 12, 1915'});

Response cookie

response.cookie(name, value, cfg)

Add a cookie to the response. Optionally the cookie can be signed.

Parameter

- name: the name of the cookie to add to the response.

- value: the value of the cookie to add to the response.

- cfg (optional): an object with any of the following properties:

- ttl (optional): the number of seconds until this cookie expires.

- path (optional): the cookie path.

- domain (optional): the cookie domain.

- secure (optional): mark the cookie as safe transport (HTTPS) only.

- httpOnly (optional): mark the cookie as HTTP(S) only.

- signed (optional): an object with any of the following properties:

- secret: a secret string to sign the cookie with.

- algorithm: hashing algorithm to sign the cookie with. Default: "sha256".

If signed is a string, it will be used as the secret instead.

If a secret is provided, a second cookie with the name name + ".sig" will be added to the response, containing the cookie's HMAC signature.

Examples

response.cookie("sid", "abcdef", {signed: "keyboardcat"});

Response send

response.send(value)

Sets the response body to the specified value. If value is a Buffer object, the content type will be set to  application/octet-stream  if not yet set. If value is a string, the content type will be set to  text/html  if not yet set. If value is an object, it will be treated as in  res.json .

Examples

response.send({"born": "December 12, 1915"});

response.send(new Buffer("some binary data"));

response.send("<html><head><title>Hello World</title></head><body></body></html>");

Response sendFile

response.sendFile(filename, options)

Sets the content of the specified file as the response body. The filename must be absolute. If no content type is yet set for the response, the response's content type will be determined automatically based on the filename extension. If no content type is known for the extension, the content type will default to  application/octet-stream .

The  options  array can be used to control the behavior of sendFile. Currently only the following option exists:

- lastModified : if set to true, the last modification date and time of the file will be returned in the  Last-Modified  HTTP header

Examples

response.sendFile('/tmp/results.json');

response.sendFile(applicationContext.fileName('image.png'), { lastModified: true });

Controlling Access to Foxx Applications

Access to Foxx applications is controlled by the regular authentication mechanisms present in ArangoDB. The server can be run with or without HTTP authentication.

If authentication is turned on, then every access to the server is authenticated via HTTP authentication. This includes Foxx applications. The global authentication can be toggled via the configuration option.

If global HTTP authentication is turned on, requests to Foxx applications will require HTTP authentication too, and only valid users present in the _users system collection are allowed to use the applications.

Since ArangoDB 1.4, there is an extra option to restrict the authentication to just system API calls, such as /_api/... and /_admin/.... This option can be turned on using the "server.authenticate-system-only" configuration option. If it is turned on, then only system API requests need authentication whereas all requests to Foxx applications and routes will not require authentication.

This is recommended if you want to disable HTTP authentication for Foxx applications but still want the general database APIs to be protected with HTTP authentication.

If you need more fine grained control over the access to your Foxx application, we built an authentication system you can use. Currently we only support cookie-based authentication, but we will add the possibility to use Auth Tokens and external OAuth providers in the near future. Of course you can roll your own authentication mechanism if you want to, and you can do it in an application-specific way if required.

To use the per-application authentication, you should first turn off the global HTTP authentication (or at least restrict it to system API calls as mentioned above). Otherwise clients will need HTTP authentication and need additional authentication by your Foxx application.

To have global HTTP authentication turned on for system APIs but turned off for Foxx, your server startup parameters should look like this:

--server.disable-authentication false --server.authenticate-system-only true

Note: During development, you may even turn off HTTP authentication completely:

--server.disable-authentication true --server.authenticate-system-only true

Please keep in mind that turning HTTP authentication off completely will allow unauthenticated access by anyone to all API functions, so do not use this is production.

Now it's time to configure the application-specific authentication. We built a small demo application to demonstrate how this works.

To use the application-specific authentication in your own app, first activate it in your controller.

Active Authentication

Controller.activateAuthentication(opts)

To activate authentication for this controller, call this function before defining any routes. In the  opts  object you can set the following keys:

- type: Currently we only support cookie, but this will change in the future

- cookieLifetime: An integer. Lifetime of cookies in seconds

- cookieName: A string used as the name of the cookie

- sessionLifetime: An integer. Lifetime of sessions in seconds

Examples

app.activateAuthentication({

type: "cookie",

cookieLifetime: 360000,

cookieName: "my_cookie",

sessionLifetime: 400,

});

Login

FoxxController#login(path, opts)

Add a route for the login. You can provide further customizations via the the options:

- usernameField and passwordField can be used to adjust the expected attributes in the post request. They default to username and password.

- onSuccess is a function that you can define to do something if the login was successful. This includes sending a response to the user. This defaults to a function that returns a JSON with user set to the identifier of the user and

- key set to the session key.

- onError is a function that you can define to do something if the login did not work. This includes sending a response to the user. This defaults to a function that sets the response to 401 and returns a JSON with error set to "Username or Password was wrong".

Both onSuccess and onError should take request and result as arguments.

Examples

app.login('/login', {

onSuccess(req, res) {

res.json({"success": true});

}

});

Logout

FoxxController#logout(path, opts)

This works pretty similar to the logout function and adds a path to your app for the logout functionality. You can customize it with a custom onSuccess and onError function:

- onSuccess is a function that you can define to do something if the logout was successful. This includes sending a response to the user. This defaults to a function that returns a JSON with message set to "logged out".

- onError is a function that you can define to do something if the logout did not work. This includes sending a response to the user. This defaults to a function that sets the response to 401 and returns a JSON with error set to "No session was found".

Both onSuccess and onError should take request and result as arguments.

Examples

app.logout('/logout', {

onSuccess(req, res) {

res.json({"message": "Bye, Bye"});

}

});

Register

FoxxController#register(path, opts)

This works pretty similar to the logout function and adds a path to your app for the register functionality. You can customize it with a custom onSuccess and onError function:

- onSuccess is a function that you can define to do something if the registration was successful. This includes sending a response to the user. This defaults to a function that returns a JSON with user set to the created user document.

- onError is a function that you can define to do something if the registration did not work. This includes sending a response to the user. This defaults to a function that sets the response to 401 and returns a JSON with error set to "Registration failed".

Both onSuccess and onError should take request and result as arguments.

You can also set the fields containing the username and password via usernameField (defaults to username) and passwordField (defaults to password). If you want to accept additional attributes for the user document, use the option acceptedAttributes and set it to an array containing strings with the names of the additional attributes you want to accept. All other attributes in the request will be ignored.

If you want default attributes for the accepted attributes or set additional fields (for example admin) use the option defaultAttributes which should be a hash mapping attribute names to default values.

Examples

app.register('/logout', {

acceptedAttributes: ['name'],

defaultAttributes: {

admin: false

}

});

Change Password

FoxxController#changePassword(route, opts)`

Add a route for the logged in user to change the password. You can provide further customizations via the the options:

- passwordField can be used to adjust the expected attribute in the post request. It defaults to password.

- onSuccess is a function that you can define to do something if the change was successful. This includes sending a response to the user. This defaults to a function that returns a JSON with notice set to "Changed password!".

- onError is a function that you can define to do something if the login did not work. This includes sending a response to the user. This defaults to a function that sets the response to 401 and returns a JSON with error set to "No session was found".

Both onSuccess and onError should take request and result as arguments.

Examples

app.changePassword('/changePassword', {

onSuccess(req, res) {

res.json({"success": true});

}

});

Restricting routes

To restrict routes, see the documentation for Documenting and Restraining the routes.

Scripts and the Job Queue

Foxx allows you to define scripts that can be executed as part of the installation and removal process, invoked manually or scheduled to run at a later time using the job queue.

To register your script, just add a scripts section to your application manifest:

{

...

"scripts": {

"setup": "scripts/setup.js",

"send-mail": "scripts/send-mail.js"

}

...

}

The scripts you define in your application manifest can be invoked from the web admin frontend or with the foxx-manager CLI:

unix>foxx-manager run /my-app-mount-point send-mail '{"to": "user@example.com", "body": "Hello"}'

You can also use the script with the Foxx job queue:

var Foxx = require('org/arangodb/foxx');

var queue = Foxx.queues.get('default');

queue.push(

{mount: '/my-app-mount-point', name: 'send-mail'},

{to: 'user@example.com', body: 'Hello'}

);

For more information the Foxx job queue, see the chapter about queues.

Script arguments and return values

If the script was invoked with any arguments, you can access them using the applicationContext.argv array.

To return data from your script, you can assign the data to module.exports as usual. Please note that this data will be converted to JSON.

Any errors raised by the script will be handled depending on how the script was invoked:

- if the script was invoked with the foxx-manager CLI, it will exit with a non-zero exit status and print the error message.

- if the script was invoked from the HTTP API (e.g. using the web admin frontend), it will return an error response using the exception's  statusCode  property if specified or 500.

- if the script was invoked from the Foxx job queue, the job's failure counter will be incremented and the job will be rescheduled or marked as failed if no attempts remain.

Examples

Let's say you want to define a script that takes two numeric values and returns the result of multiplying them:

var assert = require('assert');

var argv = applicationContext.argv;

assert.equal(argv.length, 2, 'Expected exactly two arguments');

assert.equal(typeof argv[0], 'number', 'Expected first argument to be a number');

assert.equal(typeof argv[1], 'number', 'Expected second argument to be a number');

module.exports = argv[0] * argv[1];

Life-Cycle Scripts

Foxx recognizes life-cycle scripts if they are defined and will invoke them during the installation, update and removal process of the app if you want.

The following scripts are currently recognized as life-cycle scripts:

Setup Script

The setup script will be executed without arguments during the installation of your Foxx app:

unix>foxx-manager install hello-foxx /example

Note that the app may not yet have been configured when the setup script is executed, so the applicationContext.configuration and applicationContext.dependencies may be unavailable.

The setup script is typically used to create collections your app needs or insert seed data like initial administrative user accounts and so on.

Examples

var db = require("org/arangodb").db;

var textsCollectionName = applicationContext.collectionName("texts");

// `textsCollectionName` is now the prefixed name of this app's "texts" collection.

// e.g. "example_texts" if the app has been mounted at `/example`

if (db._collection(textsCollectionName) === null) {

var collection = db._create(textsCollectionName);

collection.save({ text: "entry 1 from collection texts" });

collection.save({ text: "entry 2 from collection texts" });

collection.save({ text: "entry 3 from collection texts" });

} else {

console.log("collection '%s' already exists. Leaving it untouched.", texts);

}

Teardown Script

The teardown script will be executed without arguments during the removal of your Foxx app:

unix>foxx-manager uninstall /example

`

It can also optionally be executed before upgrading an app.

This script typically removes the collections and/or documents created by your app's setup script.

Examples

var db = require("org/arangodb").db;

var textsCollection = applicationContext.collection("texts");

if (textsCollection) {

textsCollection.drop();

}

Details on FoxxModel

The model doesn't know anything about the database. It is just a representation of the data as an JavaScript object. You can add and overwrite the methods of the prototype in your model prototype via the object you give to extend. In your model file, export the model as model.

var Foxx = require("org/arangodb/foxx");

var TodoModel = Foxx.Model.extend({

// ...

});

exports.model = TodoModel;

A Foxx Model can be initialized with an object of attributes and their values.

There's also the possibility of annotation: If you extend the model with a schema property, the model's attributes will be validated against it.

You can define attributes in the schema using the bundled joi library. For more information on the syntax see the official joi documentation.

var Foxx = require("org/arangodb/foxx");

var joi = require("joi");

var PersonModel = Foxx.Model.extend({

schema: {

name: joi.string().required(),

age: joi.number().integer(),

active: joi.boolean().default(true)

}

});

exports.model = PersonModel;

You can also use  joi.object  schemas directly:

var PersonModel = Foxx.Model.extend({

schema: joi.object().keys({

name: joi.string().required(),

age: joi.number().integer(),

active: joi.boolean().default(true)

})

});

This has two effects: On the one hand it provides documentation. If you annotated your model, you can use it in the bodyParam method for documentation. On the other hand it will influence the behavior of the constructor: If you provide an object to the constructor, it will validate its attributes and set the special errors property. This is especially useful if you want to to initialize the Model from user input. On the other hand it will set the default value for all attributes that have not been set by hand. An example:

var person = new PersonModel({

name: "Pete",

admin: true

});

person.attributes // => { name: "Pete", admin: true, active: true }

person.errors // => {admin: [ValidationError: value is not allowed]}

The following events are emitted by a model:

- beforeCreate

- afterCreate

- beforeSave

- afterSave

- beforeUpdate

- afterUpdate

- beforeRemove

- afterRemove

Equivalent events will also be emitted by the repository handling the model.

Model lifecycle:

var person = new PersonModel();

person.on('beforeCreate', function() {

var model = this;

model.fancyMethod(); // Do something fancy with the model

});

var people = new Repository(appContext.collection("people"), { model: PersonModel });

people.save(person);

// beforeCreate()

// beforeSave()

// The model is created at db

// afterSave()

// afterCreate()

people.update(person, data);

// beforeUpdate(data)

// beforeSave(data)

// The model is updated at db

// afterSave(data)

// afterUpdate(data)

people.remove(person);

// beforeRemove()

// The model is deleted at db

// afterRemove()

Extend

FoxxModel#extend(instanceProperties, classProperties)

Extend the Model prototype to add or overwrite methods. The first object contains the properties to be defined on the instance, the second object those to be defined on the prototype.

Initialize

new FoxxModel(data)

If you initialize a model, you can give it initial data as an object.

Examples

instance = new Model({

a: 1

});

Get

FoxxModel#get(name)

Get the value of an attribute

Examples

instance = new Model({

a: 1

});

instance.get("a");

Set

FoxxModel#set(name, value)

Set the value of an attribute or multiple attributes at once

Examples

instance = new Model({

a: 1

});

instance.set("a", 2);

instance.set({

b: 2

});

Has

FoxxModel#has(name)

Returns true if the attribute is set to a non-null or non-undefined value.

Examples

instance = new Model({

a: 1

});

instance.has("a"); //=> true

instance.has("b"); //=> false

isValid

model.isValid

The isValid flag indicates whether the model's state is currently valid. If the model does not have a schema, it will always be considered valid.

Errors

model.errors

The errors property maps the names of any invalid attributes to their corresponding validation error.

Attributes

model.attributes

The attributes property is the internal hash containing the model's state.

forDB

FoxxModel#forDB()

Return a copy of the model which can be saved into ArangoDB

forClient

FoxxModel#forClient()

Return a copy of the model which you can send to the client.

Details on FoxxRepository

A repository is a gateway to the database. It gets data from the database, updates it or saves new data. It uses the given model when it returns a model and expects instances of the model for methods like save. In your repository file, export the repository as repository.

var Foxx = require("org/arangodb/foxx");

var TodosRepository = Foxx.Repository.extend({

// ...

});

exports.repository = TodosRepository;

The following events are emitted by a repository:

- beforeCreate

- afterCreate

- beforeSave

- afterSave

- beforeUpdate

- afterUpdate

- beforeRemove

- afterRemove

Model lifecycle:

var person = new PersonModel();

person.on('beforeCreate', function() {

this.fancyMethod(); // Do something fancy with the model

});

var people = new Repository(appContext.collection("people"), { model: PersonModel });

people.save(person);

// beforeCreate(person)

// beforeSave(person)

// The model is created at db

// afterSave(person)

// afterCreate(person)

people.update(person, data);

// beforeUpdate(person, data)

// beforeSave(person, data)

// The model is updated at db

// afterSave(person, data)

// afterUpdate(person, data)

people.remove(person);

// beforeRemove(person)

// The model is deleted at db

// afterRemove(person)

Initialize

new FoxxRepository(collection, opts)

Create a new instance of Repository.

A Foxx Repository is always initialized with a collection object. You can get your collection object by asking your Foxx.Controller for it: the collection method takes the name of the collection (and will prepend the prefix of your application). It also takes two optional arguments:

1. Model: The prototype of a model. If you do not provide it, it will default to Foxx.Model

2. Prefix: You can provide the prefix of the application if you need it in your Repository (for some AQL queries probably)

If the Model has any static methods named after the lifecycle events, they will automatically be registered as listeners to the events emitted by this repository.

Examples

instance = new Repository(appContext.collection("my_collection"));

// or:

instance = new Repository(appContext.collection("my_collection"), {

model: MyModelPrototype

});

Example with listeners:

var ValidatedModel = Model.extend({

schema: {...}

}, {

beforeSave(modelInstance) {

if (!modelInstance.valid) {

throw new Error('Refusing to save: model is not valid!');

}

}

});

instance = new Repository(collection, {model: ValidatedModel});

Defining custom queries

You can define custom query methods using Foxx.createQuery and Foxx.Repository.extend.

For more details see the chapter on Foxx Queries.

Examples

Making a simple query in the repository and using it from the controller:

// in the repository

var Foxx = require("org/arangodb/foxx");

var TodosRepository = Foxx.Repository.extend({

getPendingItems: Foxx.createQuery(

'FOR todo IN my_todos FILTER todo.completed == false RETURN todo'

)

});

// in the controller

ctrl.get("/", function(req, res) {

req.json(todosRepository.getPendingItems());

});

It is also possible to supply parameters to the query:

// in the repository

TodosRepository.prototype.getPendingItemById = Foxx.createQuery({

query: 'FOR todo IN my_todos FILTER todo.completed == false FILTER todo._key == @id RETURN todo',

params: ['id']

});

// in the controller

ctrl.get("/:id", function(req, res) {

var id = req.params("id");

var rv = todosRepository.getPendingItemById(id);

res.json(rv);

});

The list of results can also be transformed before returning it from the repository:

// in the repository

TodosRepository.prototype.getPendingItemById = Foxx.createQuery({

query: 'FOR todo IN my_todos FILTER todo.completed == false FILTER todo._key == @id RETURN todo',

params: ['id'],

transform: function(results, extra) {

for (var i = 0; i < results.length; i++) {

results[i].extraProperty = extra;

}

}

});

// in the controller

ctrl.get("/:id", function(req, res) {

var id = req.params("id");

var extra = req.params("extra");

var rv = todosRepository.getPendingItemById(id, extra);

res.json(rv);

});

Attributes of a Repository

Collection

The wrapped ArangoDB collection object.

Model

The model of this repository. Formerly called "modelPrototype".

Model schema

The schema of this repository's model.

Prefix

The prefix of the application. This is useful if you want to construct AQL queries for example.

Defining indexes

Repository can take care of ensuring the existence of collection indexes for you. If you define indexes for a repository, instances of the repository will have access to additional index-specific methods like range or fulltext (see below).

The syntax for defining indexes is the same used in collection.ensureIndex.

Examples

var Foxx = require('org/arangodb/foxx');

var FulltextRepository = Foxx.Repository.extend({

indexes: [

{

type: 'fulltext',

fields: ['text'],

minLength: 3

}

]

});

Methods of a Repository

Adding entries to the repository

FoxxRepository#save(model)

Saves a model into the database. Expects a model. Will set the ID and Rev on the model. Returns the model.

Examples

repository.save(my_model);

Finding entries in the repository

FoxxRepository#byId(id)

Returns the model for the given ID ("collection/key") or "key".

Examples

var byIdModel = repository.byId('test/12411');

byIdModel.get('name');

var byKeyModel = repository.byId('12412');

byKeyModel.get('name');

FoxxRepository#byExample(example)

Returns an array of models for the given ID.

Examples

var myModel = repository.byExample({ amazing: true });

myModel[0].get('name');

FoxxRepository#firstExample(example)

Returns the first model that matches the given example.

Examples

var myModel = repository.firstExample({ amazing: true });

myModel.get('name');

FoxxRepository#all()

Returns an array of models that matches the given example. You can provide both a skip and a limit value.

Warning: ArangoDB doesn't guarantee a specific order in this case, to make this really useful we have to explicitly provide something to order by.

Parameter

- options (optional):

- skip (optional): skips the first given number of models.

- limit (optional): only returns at most the given number of models.

Examples

var myModel = repository.all({ skip: 4, limit: 2 });

myModel[0].get('name');

FoxxRepository#any()

Returns a random model from this repository (or null if there is none).

Examples

repository.any();

Removing entries from the repository

FoxxRepository#remove(model)

Remove the model from the repository. Expects a model.

Examples

repository.remove(myModel);

FoxxRepository#removeById(id)

Remove the document with the given ID ("collection/key") or "key". Expects an ID or key of an existing document.

Examples

repository.removeById('test/12121');

repository.removeById('12122');

FoxxRepository#removeByExample(example)

Find all documents that fit this example and remove them.

Examples

repository.removeByExample({ toBeDeleted: true });

Replacing entries in the repository

FoxxRepository#replace(model)

Find the model in the database by its _id and replace it with this version. Expects a model. Sets the revision of the model. Returns the model.

Examples

myModel.set('name', 'Jan Steemann');

repository.replace(myModel);

FoxxRepository#replaceById(id, object)

Find the item in the database by the given ID ("collection/key") or "key" and replace it with the given object's attributes.

If the object is a model, updates the model's revision and returns the model.

Examples

repository.replaceById('test/123345', myNewModel);

repository.replaceById('123346', myNewModel);

FoxxRepository#replaceByExample(example, object)

Find every matching item by example and replace it with the attributes in the provided object.

Examples

repository.replaceByExample({ replaceMe: true }, myNewModel);

Updating entries in the repository

FoxxRepository#update(model, object)

Find the model in the database by its _id and update it with the given object. Expects a model. Sets the revision of the model and updates its properties. Returns the model.

Examples

repository.update(myModel, {name: 'Jan Steeman'});

FoxxRepository#updateById(id, object)

Find an item by ID ("collection/key") or "key" and update it with the attributes in the provided object.

If the object is a model, updates the model's revision and returns the model.

Examples

repository.updateById('test/12131', { newAttribute: 'awesome' });

repository.updateById('12132', { newAttribute: 'awesomer' });

FoxxRepository#updateByExample(example, object)

Find every matching item by example and update it with the attributes in the provided object.

Examples

repository.updateByExample({ findMe: true }, { newAttribute: 'awesome' });

FoxxRepository#exists(id)

Checks whether a model with the given ID or key exists.

Examples

repository.exists(model.get('_id'));

Counting entries in the repository

FoxxRepository#count()

Returns the number of entries in this collection.

Examples

repository.count();

Index-specific repository methods

FoxxRepository#range(attribute, left, right)

Returns all models in the repository such that the attribute is greater than or equal to left and strictly less than right.

For range queries it is required that a skiplist index is present for the queried attribute. If no skiplist index is present on the attribute, the method will not be available.

Parameter

- attribute: attribute to query.

- left: lower bound of the value range (inclusive).

- right: upper bound of the value range (exclusive).

Examples

repository.range("age", 10, 13);

FoxxRepository#near(latitude, longitude, options)

Finds models near the coordinate (latitude, longitude). The returned list is sorted by distance with the nearest model coming first.

For geo queries it is required that a geo index is present in the repository. If no geo index is present, the methods will not be available.

Parameter

- latitude: latitude of the coordinate.

- longitude: longitude of the coordinate.

- options (optional):

- geo (optional): name of the specific geo index to use.

- distance (optional): If set to a truthy value, the returned models will have an additional property containing the distance between the given coordinate and the model. If the value is a string, that value will be used as the property name, otherwise the name defaults to "distance".

- limit (optional): number of models to return. Defaults to 100.

Examples

repository.near(0, 0, {geo: "home", distance: true, limit: 10});

FoxxRepository#within(latitude, longitude, radius, options)

Finds models within the distance radius from the coordinate (latitude, longitude). The returned list is sorted by distance with the nearest model coming first.

For geo queries it is required that a geo index is present in the repository. If no geo index is present, the methods will not be available.

Parameter

- latitude: latitude of the coordinate.

- longitude: longitude of the coordinate.

- radius: maximum distance from the coordinate.

- options (optional):

- geo (optional): name of the specific geo index to use.

- distance (optional): If set to a truthy value, the returned models will have an additional property containing the distance between the given coordinate and the model. If the value is a string, that value will be used as the property name, otherwise the name defaults to "distance".

- limit (optional): number of models to return. Defaults to 100.

Examples

repository.within(0, 0, 2000 * 1000, {geo: "home", distance: true, limit: 10});

FoxxRepository#fulltext(attribute, query, options)

Returns all models whose attribute attribute matches the search query query.

In order to use the fulltext method, a fulltext index must be defined on the repository. If multiple fulltext indexes are defined on the repository for the attribute, the most capable one will be selected. If no fulltext index is present, the method will not be available.

Parameter

- attribute: model attribute to perform a search on.

- query: query to match the attribute against.

- options (optional):

- limit (optional): number of models to return. Defaults to all.

Examples

repository.fulltext("text", "word", {limit: 1});

Foxx Queries

This chapter describes helpers for performing AQL queries in Foxx. For a full overview of AQL syntax and semantics see the chapter on the ArangoDB Query Language (AQL).

Raw AQL Queries

The most straightforward way to perform AQL queries is using the  db._query  API. You can learn more about this API in the chapter on invoking AQL queries.

Examples

var db = require('org/arangodb').db;

var console = require('console');

var result = db._query('RETURN 42').toArray();

console.log(result);

AQL Template Strings

ArangoDB supports ES2015-style template strings for queries using the  aqlQuery  template helper; First lets have a look at how its working, then use it for a query:

Examples

var db = require('org/arangodb').db;

var console = require('console');

var isAdmin = true;

var userCollection = applicationContext.collection('users');

var key = 'testKey';

aqlQuery`FOR c IN mycollection FILTER c._key == ${key} RETURN c._key`;

var results_in = {

"query" : "FOR c IN mycollection FILTER c._key == @value0 RETURN c._key",

"bindVars" : {

"value0" : "testKey"

}

}

var usernames = db._query(aqlQuery`

FOR user IN ${userCollection}

FILTER user.isAdmin == ${isAdmin}

RETURN user

`).toArray();

console.log('usernames:', usernames);

ArangoDB Query Builder

The ArangoDB Query Builder NPM module comes bundled with Foxx and provides a fluid API for generating complex AQL queries while avoiding raw string concatenation.

Query Builder objects can be used in any function that would normally expect an AQL string.

For a full overview of the query builder API see the project documentation.

Examples

var db = require('org/arangodb').db;

var qb = require('aqb');

var console = require('console');

var isAdmin = true;

var userCollection = applicationContext.collection('users');

var usernames = db._query(qb

.for('user')

.in(userCollection)

.filter(qb(isAdmin).eq('user.isAdmin'))

.return('user')

).toArray();

console.log('usernames:', usernames);

Foxx.createQuery

Foxx.createQuery(cfg)

Creates a query function that performs the given query and returns the result.

The returned query function optionally takes an object as its argument. If an object is provided, its properties will be used as the query's bind parameters. Any additional arguments will be passed to the transform function (or dropped if no transform function is defined).

Parameter

- cfg: an object with the following properties:

- query: an AQL query string or an ArangoDB Query Builder query object.

- params (optional): an array of parameter names.

- context (optional): an applicationContext.

- model (optional): a Foxx.Model that will be applied to the query results.

- defaults (optional): default values for the query's bind parameters. These can be overridden by passing a value for the same name to the query function.

- transform (optional): a function that will be applied to the return value.

If cfg is a string, it will be used as the value of cfg.query instead.

If a context is specified, the values of all collection bind parameters will be passed through the context's collectionName method.

Note that collection bind parameters in AQL need to be referenced with two at-signs instead of one, e.g.  @@myCollectionVar  and their parameter name needs to be prefixed with an at-sign as well, e.g.  {'@myCollectionVar': 'collection_name'} .

If params is provided, the query function will accept positional arguments instead of an object. If params is a string, it will be treated as an array containing that string.

If both model** and transform are provided, the transform function will be applied to the result array after the results have been converted into model instances. The transform function is always passed the entire result array and its return value will be returned by the query function.

Note:  Foxx.createQuery  provides a high-level abstraction around the underlying  db._query  API and provides an easy way to generate Foxx models from query results at the expense of hiding internals like query metadata (e.g. number of documents affected by a modification query). If you just want to perform AQL queries and don't need the abstractions provided by  Foxx.createQuery  you can just use the lower-level  db._query  API directly.

Examples

Basic usage example:

var query = Foxx.createQuery('FOR u IN _users RETURN u.user');

var usernames = query();

Using bind parameters:

var query = Foxx.createQuery('FOR u IN _users RETURN u[@propName]');

var usernames = query({propName: 'user'});

Using named bind parameters:

var query = Foxx.createQuery({

query: 'FOR u IN _users RETURN u[@propName]',

params: ['propName']

);

var usernames = query('user');

Using models:

var joi = require('joi');

var UserModel = Foxx.Model.extend({

schema: {

user: joi.string().required()

},

getUsername: function () {

return this.get('user');

}

});

var query = Foxx.createQuery({

query: 'FOR u IN _users RETURN u',

model: UserModel

});

var users = query();

var username = users[0].getUsername();

Using a transformation:

var query = Foxx.createQuery({

query: 'FOR u IN _users SORT u.user ASC RETURN u',

transform: function (results) {

return results[0];

}

});

var user = query(); // first user by username

Using a transformation with extra arguments:

var query = Foxx.createQuery({

query: 'FOR u IN _users SORT u.user ASC RETURN u[@propName]',

transform: function (results, uppercase) {

return uppercase ? results[0].toUpperCase() : results[0].toLowerCase();

}

});

query({propName: 'user'}, true); // username of first user in uppercase

query({propName: 'user'}, false); // username of first user in lowercase

Using a transformation with extra arguments (using positional arguments):

var query = Foxx.createQuery({

query: 'FOR u IN _users SORT u.user ASC RETURN u[@propName]',

params: ['propName'],

transform: function (results, uppercase) {

return uppercase ? results[0].toUpperCase() : results[0].toLowerCase();

}

});

query('user', true); // username of first user in uppercase

query('user', false); // username of first user in lowercase

Using a transformation with extra arguments (and no query parameters):

var query = Foxx.createQuery({

query: 'FOR u IN _users SORT u.user ASC RETURN u.user',

params: false, // an empty array would work, too

transform: function (results, uppercase) {

return uppercase ? results[0].toUpperCase() : results[0].toLowerCase();

}

});

query(true); // username of first user in uppercase

query(false); // username of first user in lowercase

Foxx Sessions

Foxx provides some convenience methods to make working with sessions easier.

Activate sessions

Enables session features for the controller.

controller.activateSessions(options)

Once sessions have been activated, a session property will be added to the request object passed to route handlers defined on the controller, which will be a saved instance of the session model provided by the session storage.

If the option autoCreateSession has not explicitly been set to false, a new session will be created for users that do not yet have an active session.

If cookie sessions are used, the session cookie will be updated after every route.

Parameter

- options (optional): an object with any of the following properties:

- sessionStorage (optional): mount point of the session storage app to use. Alternatively can be passed a session storage implementation directly. Default:  "/_system/sessions"  (the built-in session storage used by system apps).

- cookie (optional): an object with the following properties:

- name: name of the session cookie if using cookie sessions. Default:  "sid" .

- secret (optional): secret string to sign session cookies with if using cookie sessions.

- algorithm (optional): algorithm to sign session cookies with if using cookie sessions. Default:  "sha256" .

- header (optional): name of the session header if using header sessions. Default:  "X-Session-Id" .

- autoCreateSession (optional): whether a session should always be created if none exists. Default:  true .

If cookie is set to a string, its value will be used as the cookie.name instead.

If cookie is set to  true , unsigned cookies will be used with the default algorithm and cookie name.

If header is set to  true , it will be used with the default header name.

Examples

Example configuration for using signed cookies:

var controller = new FoxxController(applicationContext);

controller.activateSessions({

sessionStorage: '/_system/sessions',

cookie: {secret: 'keep-this-string-secret'}

});

Example configuration for using a header:

var controller = new FoxxController(applicationContext);

controller.activateSessions({

sessionStorage: '/_system/sessions',

header: true

});

Example configuration for using a header with a custom name:

var controller = new FoxxController(applicationContext);

controller.activateSessions({

sessionStorage: '/_system/sessions',

header: 'X-My-Session-Token'

});

Example configuration for using unsigned cookies:

var controller = new FoxxController(applicationContext);

controller.activateSessions({

sessionStorage: '/_system/sessions',

cookie: true

});

Example configuration for using unsigned cookies and a header:

var controller = new FoxxController(applicationContext);

controller.activateSessions({

sessionStorage: '/_system/sessions',

cookie: true,

header: true

});

Define a session destruction route

Defines a route that will destroy the session.

controller.destroySession(path, options)

Defines a route handler on the controller that destroys the session.

When using cookie sessions, this function will clear the session cookie (if autoCreateSession was disabled) or create a new session cookie, before calling the after function.

Parameter

- path: route path as passed to controller.get, controller.post, etc.

- options (optional): an object with any of the following properties:

- method (optional): HTTP method to handle. Default:  "post" .

- before (optional): function to execute before the session is destroyed. Receives the same arguments as a regular route handler.

- after (optional): function to execute after the session is destroyed. Receives the same arguments as a regular route handler. Default: a function that sends a {"message": "logged out"} JSON response.

Foxx Job Queues

If you have never built a Foxx App, then you can make your own first Foxx App using our cookbook.

Foxx allows defining job queues that let you perform slow or expensive actions asynchronously. These queues can be used to send e-mails, call external APIs or perform other actions that you do not want to perform directly or want to retry on failure.

--server.foxx-queues flag

If true, the Foxx queues will be available and jobs in the queues will be executed asynchronously.

The default is true. When set to  false  the queue manager will be disabled and any jobs are prevented from being processed, which may reduce CPU load a great deal. Please note that Foxx job queues are database-specific. Queues and jobs are always relative to the database in which they are created or accessed.

--server.foxx-queues-poll-interval value

The poll interval for the Foxx queues manager. The value is specified in seconds. Lower values will mean more immediate and more frequent Foxx queue job execution, but will make the queue thread wake up and query the queues more often. When set to a low value, the queue thread might cause CPU load.

The default is 1 second. If Foxx queues are not used much, then this value may be increased to make the queues thread wake up less. For the low-level functionality see the chapter on the Task Management module.

As of ArangoDB 2.6, Foxx queue job types are defined as regular Foxx scripts. To learn more about Foxx scripts, see the chapter on Foxx scripts. Function-based job types are still supported in 2.6 but have been removed in 2.7.

For an example of re-usable job types see the various mailer apps available in the Foxx app store.

Creating or updating a queue

Creates a queue with the given name and maximum number of workers.

Foxx.queues.create(name, [maxWorkers])

Returns the Queue instance for the given name. If the queue does not exist, a new queue with the given name will be created. If a queue with the given name already exists and maxWorkers is set, the queue's maximum number of workers will be updated. The queue will be created in the context of the current database.

Parameters

- name: the name of the queue to create

- maxWorkers (optional): the maximum number of workers. Default: 1

Examples

// Create a queue with the default number of workers (i.e. one)

var queue1 = Foxx.queues.create("my-queue");

// Create a queue with a given number of workers

var queue2 = Foxx.queues.create("another-queue", 2);

// Update the number of workers of an existing queue

var queue3 = Foxx.queues.create("my-queue", 10);

// queue1 and queue3 refer to the same queue

assertEqual(queue1, queue3);

Fetching an existing queue

Fetches a queue with the given name.

Foxx.queues.get(name)

Returns the Queue instance for the given name. If the queue does not exist an exception is thrown instead.

The queue will be looked for in the context of the current database.

Parameters

- name: the name of the queue to fetch

Examples

If the queue does not yet exist an exception is thrown:

Foxx.queues.get("some-queue");

// Error: Queue does not exist: some-queue

//     at ...

Otherwise the Queue instance will be returned:

var queue1 = Foxx.queues.create("some-queue");

var queue2 = Foxx.queues.get("some-queue");

assertEqual(queue1, queue2);

Deleting a queue

Deletes the queue with the given name from the database.

Foxx.queues.delete(name)

Returns true if the queue was deleted successfully. If the queue did not exist, it returns false instead. The queue will be looked for and deleted in the context of the current database.

When a queue is deleted, jobs on that queue will no longer be executed.

Deleting a queue will not delete any jobs on that queue.

Parameters

- name: the name of the queue to delete

Examples

var queue = Foxx.queues.create("my-queue");

Foxx.queues.delete("my-queue"); // true

Foxx.queues.delete("my-queue"); // false

Adding a job to a queue

Adds a job to the given queue.

The job will be added to the specified queue in the context of the current database.

Queue::push(script, data, [opts])

Returns the job id.

Parameters

- script: a job type definition, consisting of an object with the following properties:

- name: the name of the script that will be invoked.

- mount: the mount path of the app that defines the script.

- backOff (optional): either a function that takes the number of times the job has failed before as input and returns the number of milliseconds to wait before trying the job again, or the delay to be used to calculate an exponential back-off, or  0  for no delay. Default:  1000 .

- maxFailures (optional): the number of times a single run of a job will be re-tried before it is marked as  "failed" . A negative value or  Infinity  means that the job will be re-tried on failure indefinitely. Default:  0 .

- schema (optional): a Joi schema to validate a job's data against before enqueuing the job.

- preprocess (optional): a function to pre-process a job's (validated) data before serializing it in the queue.

- repeatTimes (optional): if set to a positive number, the job will be repeated this many times (not counting recovery when using maxFailures). If set to a negative number or  Infinity , the job will be repeated indefinitely. Default:  0  (no repeat).

- repeatUntil (optional): if the job is set to automatically repeat, this can be set to a timestamp in milliseconds (or  Date  instance) after which the job will no longer repeat. Setting this value to zero, a negative value or  Infinity  has no effect.

- repeatDelay (optional): if the job is set to automatically repeat, this can be set to a non-negative value to set the number of milliseconds for which the job will be delayed before it is started again. Default:  0 .

- data: the job data of the job; must be serializable to JSON.

- opts (optional): an object with any of the following properties:

- success (optional): a function to be called after the job has been completed successfully.

- failure (optional): a function to be called after the job has failed too many times.

- delayUntil (optional): a timestamp in milliseconds (or  Date  instance) until which the execution of the job should be delayed. Default:  Date.now() .

- backOff (optional): see script.backOff.

- maxFailures (optional): see script.maxFailures.

- repeatTimes (optional): see script.repeatTimes.

- repeatUntil (optional): see script.repeatUntil.

- repeatDelay (optional): see script.repeatDelay.

Note that if you pass a function for the backOff calculation, success callback or failure callback options the function will be serialized to the database as a string and therefore must not rely on any external scope or external variables.

When the job is set to automatically repeat, the failure callback will only be executed when a run of the job has failed more than maxFailures times. Note that if the job fails and maxFailures is set, it will be rescheduled according to the backOff until it has either failed too many times or completed successfully before being scheduled according to the repeatDelay again. Recovery attempts by maxFailures do not count towards repeatTimes.

The success and failure callbacks receive the following arguments:

- result: the return value of the script for the current run of the job.

- jobData: the data passed to this method.

- job: the ArangoDB document representing the job's current state.

Examples

Let's say we have an app mounted at  /mailer  that provides a script called  send-mail :

var Foxx = require("org/arangodb/foxx");

var queue = Foxx.queues.create("my-queue");

queue.push(

{mount: "/mailer", name: "send-mail"},

{to: "hello@example.com", body: "Hello world"}

);

This will not work, because log was defined outside the callback function (the callback must be serializable to a string):

var Foxx = require("org/arangodb/foxx");

var queue = Foxx.queues.create("my-queue");

var log = require("console").log; // outside the callback's function scope

queue.push(

{mount: "/mailer", name: "send-mail"},

{to: "hello@example.com", body: "Hello world"},

{success: function () {

log("Yay!"); // throws "console is not defined"

}}

);

Here's an example of a job that will be executed every 5 seconds until tomorrow:

var Foxx = require("org/arangodb/foxx");

var queue = Foxx.queues.create("my-queue");

queue.push(

{mount: "/mailer", name: "send-mail"},

{to: "hello@example.com", body: "Hello world"},

{

repeatTimes: Infinity,

repeatUntil: Date.now() + (24 * 60 * 60 * 1000),

repeatDelay: 5 * 1000

}

);

Fetching a job from the queue

Creates a proxy object representing a job with the given job id.

The job will be looked for in the specified queue in the context of the current database.

Queue::get(jobId)

Returns the Job instance for the given jobId. Properties of the job object will be fetched whenever they are referenced and can not be modified.

Parameters

- jobId: the id of the job to create a proxy object for.

Examples

var jobId = queue.push("log", "Hello World!");

var job = queue.get(jobId);

assertEqual(job.id, jobId);

Deleting a job from the queue

Deletes a job with the given job id. The job will be looked for and deleted in the specified queue in the context of the current database.

Queue::delete(jobId)

Returns true if the job was deleted successfully. If the job did not exist it returns false instead.

Fetching an array of jobs in a queue

Examples

queue.push("log", "Hello World!", {delayUntil: Date.now() + 50});

assertEqual(queue.pending("log").length, 1);

// 50 ms later...

assertEqual(queue.pending("log").length, 0);

assertEqual(queue.progress("log").length, 1);

// even later...

assertEqual(queue.progress("log").length, 0);

assertEqual(queue.complete("log").length, 1);

Fetching an array of pending jobs in a queue

Queue::pending([script])

Returns an array of job ids of jobs in the given queue with the status  "pending" , optionally filtered by the given job type. The jobs will be looked for in the specified queue in the context of the current database.

Parameters

- script (optional): an object with the following properties:

- name: name of the script.

- mount: mount path of the app defining the script.

Fetching an array of jobs that are currently in progress

Queue::progress([script])

Returns an array of job ids of jobs in the given queue with the status  "progress" , optionally filtered by the given job type. The jobs will be looked for in the specified queue in the context of the current database.

Parameters

- script (optional): an object with the following properties:

- name: name of the script.

- mount: mount path of the app defining the script.

Fetching an array of completed jobs in a queue

Queue::complete([script])

Returns an array of job ids of jobs in the given queue with the status  "complete" , optionally filtered by the given job type. The jobs will be looked for in the specified queue in the context of the current database.

Parameters

- script (optional): an object with the following properties:

- name: name of the script.

- mount: mount path of the app defining the script.

Fetching an array of failed jobs in a queue

Queue::failed([script])

Returns an array of job ids of jobs in the given queue with the status  "failed" , optionally filtered by the given job type. The jobs will be looked for in the specified queue in the context of the current database.

Parameters

- script (optional): an object with the following properties:

- name: name of the script.

- mount: mount path of the app defining the script.

Fetching an array of all jobs in a queue

Queue::all([script])

Returns an array of job ids of all jobs in the given queue, optionally filtered by the given job type. The jobs will be looked for in the specified queue in the context of the current database.

Parameters

- script (optional): an object with the following properties:

- name: name of the script.

- mount: mount path of the app defining the script.

Aborting a job

Aborts a non-completed job.

Job::abort()

Sets a job's status to  "failed"  if it is not already  "complete" , without calling the job's onFailure callback.

Foxx console

Foxx injects a console object into each Foxx app that allows writing log entries to the database in addition to the ArangoDB log file and querying them from within the app itself.

The console object supports the CommonJS Console API found in Node.js and modern browsers, while also providing some ArangoDB-specific additions.

ArangoDB also provides the console module which only supports the CommonJS Console API and only writes log entries to the ArangoDB log.

When working with transactions, keep in mind that the Foxx console will attempt to write to the  _foxxlog  system collection. This behaviour can be disabled using the  setDatabaseLogging  method if you don't want to explicitly allow writing to the log collection during transactions or for performance reasons.

Logging

Logging console messages

Write an arbitrary message to the app's log.

console.log([format,] ...parameters)

Applies  util.format  to the arguments and writes the resulting string to the app's log.

If the first argument is not a formatting string or any of the additional arguments are not used by the formatting string, they will be concatenated to the result string.

Examples

console.log("%s, %s!", "Hello", "World"); // => "Hello, World!"

console.log("%s, World!", "Hello", "extra"); // => "Hello, World! extra"

console.log("Hello,", "beautiful", "world!"); // => "Hello, beautiful world!"

console.log(1, 2, 3); // => "1 2 3"

Logging with different log levels

The console object provides additional methods to log messages with different log levels:

- console.debug  for log level DEBUG

- console.info  for log level INFO

- console.warn  for log level WARN

- console.error  for log level ERROR

By default,  console.log  uses log level INFO, making it functionally equivalent to  console.info . Other than the log level, all of these methods behave identically to  console.log .

The built-in log levels are:

- -200: TRACE

- -100: DEBUG

- 0: INFO

- 100: WARN

- 200: ERROR

Logging with timers

You can start and stop timers with labels.

console.time(label)  /  console.timeEnd(label)

Passing a label to  console.time  starts the timer. Passing the same label to  console.timeEnd  ends the timer and logs a message with the label and the expired time in milliseconds.

Calling  console.timeEnd  with an invalid label or calling it with the same label twice (without first starting a new timer with the same label) results in an error.

By default, the timing messages will be logged with log level INFO.

Examples

console.time('do something');

// ... more code ...

console.time('do more stuff');

// ... even more code ...

console.timeEnd('do more stuff'); // => "do more stuff: 23ms"

// ... a little bit more ....

console.timeEnd('do something'); // => "do something: 52ms"

Don't do this:

console.timeEnd('foo'); // fails: label doesn't exist yet.

console.time('foo'); // works

console.timeEnd('foo'); // works

console.timeEnd('foo'); // fails: label no longer exists.

Logging stack traces

You can explicitly log a message with a stack trace.

console.trace(message)

This creates a stack trace with the given message.

By default the stack traces will be logged with a log level of TRACE.

Examples

console.trace('Hello');

/* =>

Trace: Hello

at somefile.js:1:1

at ...

*/

Logging assertions

This creates an assertion that will log an error if it fails.

console.assert(statement, message)

If the given statement is not true (e.g. evaluates to  false  if treated as a boolean), an  AssertionError  with the given message will be created and its stack trace will be logged.

By default, the stack trace will be logged with a log level of ERROR and the error will be discarded after logging it (instead of being thrown).

Examples

console.assert(2 + 2 === 5, "I'm bad at maths");

/* =>

AssertionError: I'm bad at maths

at somefile.js:1:1

at ...

*/

Inspecting an object

This logs a more detailed string representation of a given object.

console.dir(value)

The regular logging functions try to format messages as nicely and readable as possible. Sometimes that may not provide you with all the information you actually want.

The dir method instead logs the result of  util.inspect .

By default the message will be logged with the log level INFO.

Examples

console.log(require('org/arangodb').db); // => '[ArangoDatabase "_system"]'

console.dir(require('org/arangodb').db);

/* =>

{ [ArangoDatabase "_system"]

_version: [Function: _version],

_id: [Function: _id],

...

}

*/

Custom log levels

This lets you define your own log levels.

console.custom(name, value)

If you need more than the built-in log levels, you can easily define your own.

This method returns a function that logs messages with the given log level (e.g. an equivalent to  console.log  that uses your custom log level instead).

Parameter

- name: name of the log level as it appears in the database, usually all-uppercase

- value (optional): value of the log level. Default:  50

The value is used when determining whether a log entry meets the minimum log level that can be defined in various places. For a list of the built-in log levels and their values see the section on logging with different log levels above.

Preventing entries from being logged

You can define a minimum log level entries must match in order to be logged.

console.setLogLevel(level)

The level can be a numeric value or the case-sensitive name of an existing log level.

Any entries with a log level lower than the given value will be discarded silently.

This can be helpful if you want to toggle logging diagnostic messages in development mode that should not be logged in production.

The default log level is set to  -999 . For a list of built-in log levels and their values see the section on logging with different log levels.

Enabling extra stack traces

You can toggle the logging of stack trace objects for every log entry.

console.setTracing(trace)

If trace is set to  true , all log entries will be logged with a parsed stack trace as an additional  stack  property that can be useful for identifying where the entry originated and how the code triggering it was called.

Because this results in every logging call creating a stack trace (which may have a significant performance impact), this option is disabled by default.

Disabling logging to the ArangoDB console

You can toggle whether logs should be written to the ArangoDB console.

console.setNativeLogging(nativeLogging)

If nativeLogging is set to  false , log entries will not be logged to the ArangoDB console (which usually writes to the file system).

Disabling logging to the database

You can toggle whether logs should be written to the database.

console.setDatabaseLogging(databaseLogging)

If databaseLogging is set to  false , log entries will not be logged to the internal  _foxxlog  collection.

This is only useful if logging to the ArangoDB console is not also disabled.

Enabling assertion errors

You can toggle whether console assertions should throw if they fail.

console.setAssertThrows(assertThrows)

If assertThrows is set to  true , any failed assertions in  console.assert  will result in the generated error being thrown instead of being discarded after it is logged.

By default, this setting is disabled.

Changing the default log levels

Most of the logging methods have an implied log level that is set to a reasonable default. If you would like to have them use different log levels, you can easily change them.

- console.log.level  defaults to INFO

- console.dir.level  defaults to INFO

- console.time.level  defaults to INFO

- console.trace.level  defaults to TRACE

- console.assert.level  defaults to ERROR

To use different log levels, just set these properties to the case-sensitive name of the desired log level.

Examples

console.log('this uses "INFO"');

console.log.level = 'WARN';

console.log('now it uses "WARN"');

Known custom levels work, too:

console.custom('BANANA', 23);

console.log.level = 'BANANA';

console.log('now it uses "BANANA"');

Unknown levels result in an error:

console.log.level = 'POTATO';

console.log('this throws'); // => Error: Unknown log level: POTATO

Querying a Foxx app's log entries

As the log entries are logged to a collection in the database, you can easily query them in your own application.

The logs object

The logs object can be found on the console itself:

console.logs

It provides three easy methods for querying the log entries:

- logs.list([opts])

- logs.searchByMessage(message, [opts])

- logs.searchByFileName(fileName, [opts])

Each method takes an optional  opts  argument, which can be an object with any of the following properties:

- startTime: the oldest time to include (in milliseconds). Default: 2 hours ago.

- endTime: the most recent time to include (in milliseconds).

- level: only return entries with this log level (name or value).

- minLevel: only return entries with this log level or above (name or value).

- sort: sorting direction of the result (ASC or DESC by time). Default: ASC.

- limit: maximum number of entries to return.

- offset: if limit is set, skip this many entries.

The default value for startTime can be changed by overriding  logs.defaultMaxAge  with a different time offset in milliseconds.

Search by message

This lists all log entries with messages that contain the given token.

logs.searchByMessage(message, [opts])

This works like  logs.list  except it only returns log entries containing the given message part in their message.

Search by file name

This lists all log entries with stack traces that contain the given token.

logs.searchByFileName(fileName, [opts])

This works like  logs.list  except it only returns log entries containing the given fileName part in one of the file names of their stack trace.

This method can only be used if the console has tracing enabled. See the section on enabling extra stack traces.

Note that entries that were logged while tracing was not enabled can not be found with this method because they don't have any parsed stack traces associated with them. This method does not search the log entries messages for the file name, so entries generated by  console.assert  or  console.trace  are not treated differently.

The Manifest File

In the manifest.json you define the components of your application. The content is a JSON object with the following attributes (not all attributes are required though):

- author: The author name

- configuration: Define your app's configuration parameters.

- contributors: An array containing objects, each represents a contributor (with name and optional email)

- controllers: Map routes to FoxxControllers

- defaultDocument: The default document when the application's root ( / ) is called (defaults to  "index.html" )

- dependencies: Map names to Foxx apps

- description: A short description of the application (Meta information)

- engines: Should be an object with arangodb set to the ArangoDB version your Foxx app is compatible with

- exports: Map names to Foxx exports

- files: Deliver files

- isSystem: Mark an application as a system application

- keywords: An array of keywords to help people find your Foxx app

- lib: Base path for all required modules

- license: Short form of the license (MIT, GPL...)

- name: Name of the application (Meta information)

- repository: An object with information about where you can find the repository: type and url

- scripts: An object with script names mapped to filenames, e.g. your app's setup and teardown scripts

- tests: An array of names of files containing mocha tests for the Foxx app.

- thumbnail: Path to a thumbnail that represents the application (Meta information)

- version: Current version of the application (Meta information)

If you install an application using the Foxx manager or are using the development mode, your manifest will be checked for completeness and common errors. You should have a look at the server log files after changing a manifest file to get informed about potential errors in the manifest.

A more complete example for a Manifest file:

{

"name": "my_website",

"version": "1.2.1",

"description": "My Website with a blog and a shop",

"thumbnail": "images/website-logo.png",

"engines": {

"arangodb": "^2.7.0"

},

"configuration": {

"currency": {

"description": "Currency symbol to use for prices in the shop.",

"default": "$",

"type": "string"

}

},

"controllers": {

"/blog": "apps/blog.js",

"/shop": "apps/shop.js"

},

"exports": "index.js",

"lib": "lib",

"files": {

"/images": "images"

},

"scripts": {

"setup": "scripts/setup.js",

"teardown": "scripts/teardown.js",

"some-maintenance-script": "scripts/prune-comments.js"

},

"tests": [

"test/**",

"test-*.js"

],

"dependencies": {

"sessions": "sessions@^1.0.0",

"systemUsers": "users",

"mailer": {

"name": "mailer-postmark",

"version": "*",

"required": false

}

}

}

The setup and teardown scripts

You can provide a path to a JavaScript file that prepares ArangoDB for your application (or respectively removes it entirely). Use the setup script to create all collections your application needs and fill them with initial data if you want to. Use the teardown script to remove all collections you have created.

Note: the setup script is called on each request in the development mode. If your application needs to set up specific collections, you should always check in the setup script whether they are already there.

The teardown script is called when an application is uninstalled. It is good practice to drop any collections in the teardown script that the application used exclusively, but this is not enforced. Maybe there are reasons to keep application data even after removing an application. It's up to you to decide what to do.

Mocha tests

You can provide test cases for your Foxx app using the mocha test framework and an assertion library like expect.js or chai (or even the built-in assert module).

The tests array lists the relative paths of all mocha tests of your Foxx app. In addition to regular paths, the array can also contain any patterns supported by the minimatch module, for example:

- glob matching:  ./tests/*.js  will match all JS files in the folder "tests"

- globstar matching:  ./tests/**  will match all files and subfolders of the folder "tests"

- brace expansion:  ./tests/{a,b,c}.js  will match the files "a.js", "b.js" and "c.js"

For more information on the supported patterns see the minimatch documentation.

Configuration parameters

Foxx apps can define configuration parameters to make them more re-usable.

The configuration object maps names to configuration parameters:

- The key is the name under whicht the parameter will be available on your applicationContext.configuration object

- The value is a parameter definition.

The parameter definition can have the following properties:

- description: a human readable description of the parameter.

- type: the type of the configuration parameter. Default:  "string" .

- default: the default value of the configuration parameter.

- required: whether the parameter is required. Default:  true

The type can be any of the following:

- integer or int: any finite integer number.

- boolean or bool:  true  or  false .

- number: any finite decimal or integer number.

- string: any string value.

- json: any well-formed JSON value.

- password: like string but will be displayed as a masked input field in the web frontend.

If the configuration has parameters that do not specify a default value, you need to configure the app before it becomes active. In the meantime a fallback application will be mounted that responds to all requests with a HTTP 500 status code indicating a server-side error.

The configuration parameters of a mounted app can be adjusted from the admin frontend by clicking the Configuration button in the app details or using the configure command of the foxx-manager command-line utility.

Defining dependencies

Foxx apps can depend on other Foxx apps to be installed on the same server.

The dependencies object maps aliases to Foxx apps:

- The key is the name under which the dependency's exports will be available on your applicationContext.dependencies object.

- The value is a dependency definition.

The dependency definition is an object with any of the following properties:

- name (optional): the name of the Foxx app this app depends on.

- version (Default:  "*" ): a semver version or version range of the Foxx app this app depends on.

- required (Default:  true ): whether the dependency is required for this app to be usable or not.

Alternatively the dependency definition can be a string using any of the following formats:

- *  will allow using any app to be used to meet the dependency.

- sessions  or  sessions:*  will match any app with the name  sessions  (such as the sessions app in the Foxx application store).

- sessions:1.0.0  will match the version  1.0.0  of any app with the name  sessions .

Instead of using a specific version number, you can also use any expression supported by the semver module.

Currently the dependency definition names and versions are not enforced in ArangoDB but this may change in a future version.

If an app declares any required dependencies, you need to fulfill its dependencies before it becomes active. In the meantime a fallback application will be mounted that responds to all requests with a HTTP 500 status code indicating a server-side error.

The dependencies of a mounted app can be adjusted from the admin frontend by clicking the Dependencies button in the app details or using the set-dependencies command of the foxx-manager command-line utility.

For more information on dependencies see the chapter on Foxx Exports.

Defining controllers

Controllers can be defined as an object mapping routes to file names:

- The key is the route you want to mount at

- The value is the path to the JavaScript file containing the FoxxController you want to mount

You can add multiple controllers in one manifest this way.

If controllers is set to a string instead, it will be treated as the value with the key being implicitly set to  "/"  (i.e. the root of your app's mount point).

In other words, the following:

{

"controllers": "my-controllers.js"

}

is equivalent to this:

{

"controllers": {

"/": "my-controllers.js"

}

}

The files

Deliver all files in a certain folder without modifying them. You can deliver text files as well as binaries:

"files": {

"/images": "images"

}

Working with Foxx exports

Instead of (or in addition to) defining controllers, Foxx apps can also define exports.

Foxx exports are not intended to replace regular NPM modules. They simply allow you to make your app's collections and applicationContext available in other Foxx apps or bundling ArangoDB-specific modules in re-usable Foxx apps.

Define an export module

In order to export modules in a Foxx app you need to list the files in your manifest:

{

"name": "foxx_exports_example",

"version": "1.0.0",

"description": "Demonstrates Foxx exports.",

"exports": {

"doodads": "./doodads.js",

"anotherModule": "./someOtherFilename.js"

},

"controllers": {

"/etc": "./controllers.js"

}

}

The file doodads.js in the app's base path could look like this:

var Foxx = require('org/arangodb/foxx');

var Doodad = Foxx.Model.extend({});

var doodadRepo = new Foxx.Repository(

applicationContext.collection('doodads'),

{model: Doodad}

);

exports.repo = doodadRepo;

exports.model = Doodad;

// or simply

module.exports = {

repo: doodadRepo,

model: Doodad

};

This module would then export the name "repo" bound to the variable doodads as well as the name "model" bound to the Doodad model.

Note: The applicationContext is available to your Foxx exports just like in your Foxx controllers.

Note: Node.js style exports by assigning values directly to the  module.exports  property are supported, too.

Import from another app

In order to import from another app, you need to know where the app is mounted.

Let's say we have mounted the example app above at /my-doodads. We could now access the app's exports in another app like so:

var Foxx = require('org/arangodb/foxx');

var doodads = Foxx.requireApp('/my-doodads').doodads;

var Doodad = doodads.model;

var doodadRepo = doodads.repo;

// use the imported model and repository

var myDoodad = new Doodad();

doodadRepo.save(myDoodad);

Default exports

If you want more control over the object other apps receive when they load your app using Foxx.requireApp, you can specify a single filename instead of an object in your manifest file:

{

"name": "foxx_exports_example",

"version": "1.0.0",

"description": "Demonstrates Foxx exports.",

"exports": "./exports.js",

"controllers": {

"/etc": "./controllers.js"

}

}

To replicate the same behavior as in the earlier example, the file exports.js could look like this:

exports.doodads = require('./doodads');

exports.anotherModule = require('./someOtherFilename');

// or Node.js-style

module.exports = {

doodads: require('./doodads'),

anotherModule: require('./someOtherFilename')

};

Let's assume the file exports.js exports something else instead:

module.exports = function () {

return "Hello!";

};

Assuming the app is mounted at  /hello , we could now use it in other apps like this:

var Foxx = require('org/arangodb/foxx');

var helloExport = Foxx.requireApp('/hello');

var greeting = helloExport(); // "Hello!"

Managing app dependencies

As of ArangoDB 2.6 you can define Foxx app dependencies of your Foxx app in your manifest.

Let's say you want to use the  foxx_exports_example  app from earlier in your app. Just add it to your own manifest like this:

{

"name": "foxx_dependencies_example",

"version": "1.0.0",

"dependencies": {

"exportsExample": {

"name": "foxx_exports_example",

"version": "^1.0.0"

}

}

}

Instead of specifying the exact mount point of the app you depend on, you can now refer to it using the name  exportsExample :

Before:

var doodads = Foxx.requireApp(applicationContext.configuration.exportsExampleMountPath).doodads;

After:

var doodads = applicationContext.dependencies.exportsExample.doodads;

If an app declares any dependencies, you need to fulfill its dependencies before it becomes active. In the meantime a fallback application will be mounted that responds to all requests with a HTTP 500 status code indicating a server-side error.

The dependencies of a mounted app can be adjusted from the admin frontend by clicking the Dependencies button in the app details or using the set-dependencies command of the foxx-manager command-line utility.

Foxx API Documentation

In addition to viewing the API documentation of any Foxx app in the admin frontend, you can also mount the API documentation inside your own Foxx app. This allows you to serve your documentation to users without having to give them access to the admin frontend or any other parts of ArangoDB.

Mounting the API documentation

Controller.apiDocumentation(path, [opts])

Mounts the API documentation (Swagger) at the given  path .

Note that the  path  can use path parameters as usual but must not use any wildcard ( * ) or optional ( :name? ) parameters.

The optional opts can be an object with any of the following properties:

- before: a function that will be executed before a request to this endpoint is processed further.

- appPath: the mount point of the app for which documentation will be shown. Default: the mount point of the active app.

- indexFile: file path or file name of the Swagger HTML file. Default:  "index.html" .

- swaggerJson: file path or file name of the Swagger API description JSON file or a function  swaggerJson(req, res, opts)  that sends a Swagger API description in JSON. Default: the built-in Swagger description generator.

- swaggerRoot: absolute path that will be used as the path path for any relative paths of the documentation assets, swaggerJson file and the indexFile. Default: the built-in Swagger distribution.

If opts is a function, it will be used as the value of opts.before.

If opts.before returns  false , the request will not be processed further.

If opts.before returns an object, any properties will override the equivalent properties of opts for the current request.

Of course all before, after or around functions defined on the controller will also be executed as usual.

Examples

controller.apiDocumentation('/my/dox');

A request to  /my/dox  will be redirect to  /my/dox/index.html , which will show the API documentation of the active app.

controller.apiDocumentation('/my/dox', function (req, res) {

if (!req.session.get('uid')) {

res.status(403);

res.json({error: 'only logged in users may see the API'});

return false;

}

return {appPath: req.parameters.mount};

});

A request to  /my/dox/index.html?mount=/_admin/aardvark  will show the API documentation of the admin frontend (mounted at  /_admin/aardvark ). If the user is not logged in, the error message will be shown instead.

Running Foxx in production

This chapter will explain to you how to run Foxx in a production environment. Be it a microservice, an API with an user interface or an internal library.

Before reading this chapter you should make sure to at least read one of the install sections beforehand to get a good staring point.

Production Mode

At first we will introduce the production mode and describe its side effects. This is the default mode for all freshly installed Foxxes. It will compute the routing of a Foxx only once and afterwards use the cached version. This mode is not suitable for development, it has less error output and does not react to source file changes.

Read More

Debugging

Next you will learn about the restricted debugging mechanisms. The focus of this mode is to seal your source files and never send any stacktraces unintentionally. However there might still be unknown bugs in your Foxx, these will still be logged into the server-side log.

Read More

Upgrade to new versions

If you want to update to a new version of your application the next chapter is important for you:

New Versions

Application Production Mode

This chapter describes the production mode for Foxx applications. This is the default mode for all newly installed apps and should always be used in a production environment.

Activation

Activating the production mode is done with a single command:

unix> foxx-manager production /example

Activated production mode for Application hello-foxx version 1.5.0 on mount point /example

Now the app will not be listed in listDevelopment:

unix> foxx-manager listDevelopment

Mount       Name          Author          Description                                 Version    Development

---------   -----------   -------------   -----------------------------------------   --------   ------------

---------   -----------   -------------   -----------------------------------------   --------   ------------

0 application(s) found

Effects

For a Foxx application in production mode the following effects apply:

Application Caching The first time an application is requested it's routing is computed and cached. It is guaranteed that subsequent requests to this application will get the cached routing. That implies that changing the sources of an app will not have any effect. It also guarantees that requests will be answered at the highest possible speed. The cache might be flushed whenever ArangoDB is restarted or an application in development mode is requested.

Only internal Debugging information Whenever the application is requested and an unhandled error occurs only a generic error message will be returned. Stack traces are never delivered via the public API. For more information see the Debugging section.

Considerations for production environments

So you have created your server side application utilizing Foxx services as their backend. To get optimal performance you may want to implement an HTTP connection pool using keepalive.

You may even consider to implement non blocking HTTP requests to save resources on your connection pool.

Available Debugging mechanisms

We are talking about the production mode for Foxx. We assume that the development process is over and you have a (third-party) client using your API. In this case you will most likely want to expose stacktraces or detailed error information to the requesting client. This of course restricts the debugging mechanisms for the client and requires to consult the logs. Nevertheless the client should see if there is an internal error occurring therefore all Foxxes will have the behavior described in the following.

Right now we assume you have the production mode enabled (default after install)

unix>foxx-manager production /example

Activated production mode for Application hello-foxx version 1.5.0 on mount point /example

Errors during install

Malformed Foxx applications cannot be installed via the Foxx-manager. You will get detailed error information if you try to. However you might create the following situation:

1. Set a Foxx to development mode

2. Make it malformed

3. Set it to production mode

In this case all routes of this Foxx will create a general, HTML page for third party clients stating that there was an internal error. This page does not contain any information specific for your Foxx.

Errors in routes

If you have an unhandled error in one of your routes the error message will be returned together with an HTTP status code 500. It will not contain the stacktrace of your error.

unix>curl -X GET http://localhost:8529/_db/_system/example/failed

HTTP/1.1 500 Internal Error

Server: ArangoDB

Connection: Keep-Alive

Content-Type: application/json

Content-Length: 27

{"error":"Unhandled Error"}

Errors in logs

Independent from the errors presented in the routes on requests Foxxes will always log errors to the log-file if caught by the default error handlers. The log entries will always contain stacktraces and error messages:

ERROR Error in foxx route '{ "match" : "/failed", "methods" : [ "get" ] }': 'Unhandled Error', Stacktrace: Error: Unhandled Error

ERROR   at fail (js/apps/_db/_system/example/APP/app.js:279:13)

ERROR   at js/apps/_db/_system/example/APP/app.js:284:5

ERROR   at Object.res.action.callback (./js/server/modules/org/arangodb/foxx/internals.js:108:5)

ERROR   at ./js/server/modules/org/arangodb/foxx/routing.js:346:19

ERROR   at execute (./js/server/modules/org/arangodb/actions.js:1291:7)

ERROR   at next (./js/server/modules/org/arangodb/actions.js:1308:7)

ERROR   at [object Object]:386:5

ERROR   at execute (./js/server/modules/org/arangodb/actions.js:1291:7)

ERROR   at routeRequest (./js/server/modules/org/arangodb/actions.js:1312:3)

ERROR   at foxxRouting (./js/server/modules/org/arangodb/actions.js:1082:7)

ERROR   at execute (./js/server/modules/org/arangodb/actions.js:1291:7)

ERROR   at Object.routeRequest (./js/server/modules/org/arangodb/actions.js:1312:3)

ERROR   at Function.actions.defineHttp.callback (js/actions/api-system.js:51:15)

ERROR

Upgrade to a new version

In this chapter we want to cover the case where you have a Foxx in production and want to upgrade it to a new version. In this specific case you do not want to execute the setup and teardown scripts. A simple uninstall / install procedure would first trigger both scripts and would trigger a very short downtime of your app. Therefore the Foxx-manager offers an update procedure, which acts similar to install, but overwrites the existing Foxx. In addition it does neither trigger setup nor teardown.

unix> foxx-manager update itzpapalotl:1.0.0 /example

Application itzpapalotl version 1.0.0 installed successfully at mount point /example

Also you will not experience any downtime of your Foxx. One thread will be responsible to update to the newer version, all other threads will continue to serve the old version. As soon as the update thread is finished rerouting of all other threads is scheduled as well and eventually they will converge to the newest version of the app: All requests issued before the rerouting scheduling will be answered with the old version, all requests issued after will be answered with the new version.

Foxx in the Cluster

This chapter is about running Foxx in a cluster of ArangoDBs. There is no major conceptual difference from running Foxx in a cluster compared to running Foxx on a single machine. It is just a couple of minor things to consider.

Installing Foxx on a cluster

The Foxx installation process is agnostic of the cluster architecture. You simply use the foxx manager as described earlier and connect it to any of your coordinators. This will automatically spread the Foxx across all coordinators in the cluster.

Development Mode

Within a Cluster the development mode of Foxx is totally unsupported. The spreading of information will not be executed on each change on disc. Please develop your Foxx in a single server development environment and upload it to the cluster in production mode.

Restrictions

The code you write in Foxx should be independent from the server it is running on. This means you should avoid using the file-system module  fs  as this is writing on the local file system. This will get out of sync if you use more than one server. If you install 3rd party libraries please make sure they do not rely on the local file system. All other operations should be save and the code for single server and cluster is identical.

Administrative Scripts in Cluster

If you want to scale the dataset you access via Foxx you have to take sharding into account. This results in minor changes to your setup and teardown scripts. These will be executed on all servers you deploy your Foxx to, hence you have to consider that they will be executed multiple times. Also the setup for collections now requires you to give the amount of shards for the collection (recommended is the amount of servers squared).

Setup script

This script has to take into consideration that collections might have been installed by other servers already. Also you have to give the amount of shards (9 in this example):

var console = require("console");

var arangodb = require("org/arangodb");

var db = arangodb.db;

var texts = applicationContext.collectionName("texts");

if (db._collection(texts) === null) {

// This is the first one running the script

var collection = db._create(texts, {

numberOfShards: 9

});

collection.save({ text: "entry 1 from collection texts" });

collection.save({ text: "entry 2 from collection texts" });

collection.save({ text: "entry 3 from collection texts" });

}

else {

console.log("collection '%s' already exists. Leaving it untouched.", texts);

}

Teardown script

Also this script has to take into account that it might be run several times. You can also omit the teardown execution with passing  teardown:false  options to the uninstall process.

unix>foxx-manager uninstall /example teardown=false

The teardown script just has to check if the collection is not yet dropped:

var arangodb = require("org/arangodb");

var db = arangodb.db;

var texts = applicationContext.collectionName("texts");

var collection = db._collection(texts);

if (collection !== null) {

// Not yet dropped. Drop it now

collection.drop();

}

ArangoDB's Actions

Introduction to User Actions

In some ways the communication layer of the ArangoDB server behaves like a Web server. Unlike a Web server, it normally responds to HTTP requests by delivering JSON objects. Remember, documents in the database are just JSON objects. So, most of the time the HTTP response will contain a JSON document from the database as body. You can extract the documents stored in the database using HTTP GET. You can store documents using HTTP POST.

However, there is something more. You can write small snippets - so called actions - to extend the database. The idea of actions is that sometimes it is better to store parts of the business logic within ArangoDB.

The simplest example is the age of a person. Assume you store information about people in your database. It is an anti-pattern to store the age, because it changes every now and then. Therefore, you normally store the birthday and let the client decide what to do with it. However, if you have many different clients, it might be easier to enrich the person document with the age using actions once on the server side.

Or, for instance, if you want to apply some statistics to large data-sets and you cannot easily express this as query. You can define a action instead of transferring the whole data to the client and do the computation on the client.

Actions are also useful if you want to restrict and filter data according to some complex permission system.

The ArangoDB server can deliver all kinds of information, JSON being only one possible format. You can also generate HTML or images. However, a Web server is normally better suited for the task as it also implements various caching strategies, language selection, compression and so on. Having said that, there are still situations where it might be suitable to use the ArangoDB to deliver HTML pages - static or dynamic. A simple example is the built-in administration interface. You can access it using any modern browser and there is no need for a separate Apache or IIS.

In general you will use Foxx to easily extend the database with business logic. Foxx provides an simple to use interface to actions.

The following sections will explain the low-level actions within ArangoDB on which Foxx is built and show how to define them. The examples start with delivering static HTML pages - even if this is not the primary use-case for actions. The later sections will then show you how to code some pieces of your business logic and return JSON objects.

The interface is loosely modeled after the JavaScript classes for HTTP request and responses found in node.js and the middleware/routing aspects of connect.js and express.js.

Note that unlike node.js, ArangoDB is multi-threaded and there is no easy way to share state between queries inside the JavaScript engine. If such state information is required, you need to use the database itself.

A Hello World Example

The client API or browser sends a HTTP request to the ArangoDB server and the server returns a HTTP response to the client. A HTTP request consists of a method, normally GET or POST when using a browser, and a request path like /hello/world. For a real Web server there are a zillion of other thing to consider, we will ignore this for the moment. The HTTP response contains a content type, describing how to interpret the returned data, and the data itself.

In the following example, we want to define an action in ArangoDB, so that the server returns the HTML document

<html>

<body>

Hello World

</body>

</html>

if asked GET /hello/world.

The server needs to know what function to call or what document to deliver if it receives a request. This is called routing. All the routing information of ArangoDB is stored in a collection _routing. Each entry in this collections describes how to deal with a particular request path.

For the above example, add the following document to the _routing collection:

arangosh> db._routing.save({

........>  url: {

........>    match: "/hello/world"

........>  },

........>  content: {

........>    contentType: "text/html",

........>    body: "<html><body>Hello World</body></html>"

........>  }

........> });

show execution results

In order to activate the new routing, you must either restart the server or call the internal reload function.

arangosh> require("internal").reloadRouting()

Now use the browser and access http:// localhost:8529/hello/world

You should see the Hello World in our browser:

shell> curl --dump - http://localhost:8529/hello/world

HTTP/1.1 200 OK

content-type: text/html

"Hello World"

Matching a URL

There are a lot of options for the url attribute. If you define different routing for the same path, then the following simple rule is applied in order to determine which match wins: If there are two matches, then the more specific wins. I. e, if there is a wildcard match and an exact match, the exact match is preferred. If there is a short and a long match, the longer match wins.

Exact Match

If the definition is

{

url: {

match: "/hello/world"

}

}

then the match must be exact. Only the request for /hello/world will match, everything else, e. g. /hello/world/my or /hello/world2, will not match.

The following definition is a short-cut for an exact match.

{

url: "/hello/world"

}

Note: While the two definitions will result in the same URL matching, there is a subtle difference between them:

The former definition (defining url as an object with a match attribute) will result in the URL being accessible via all supported HTTP methods (e.g. GET, POST, PUT, DELETE, ...), whereas the latter definition (providing a string url attribute) will result in the URL being accessible via HTTP GET and HTTP HEAD only, with all other HTTP methods being disabled. Calling a URL with an unsupported or disabled HTTP method will result in an HTTP 501 (not implemented) error.

Prefix Match

If the definition is

{

url: {

match: "/hello/world/*"

}

}

then the match can be a prefix match. The requests for /hello/world, /hello/world/my, and /hello/world/how/are/you will all match. However /hello/world2 does not match. Prefix matches within a URL part, i. e. /hello/world*, are not allowed. The wildcard must occur at the end, i. e.

/hello/*/world

is also disallowed.

If you define two routes

{ url: { match: "/hello/world/*" } }

{ url: { match: "/hello/world/emil" } }

then the second route will be used for /hello/world/emil because it is more specific.

Parameterized Match

A parameterized match is similar to a prefix match, but the parameters are also allowed inside the URL path.

If the definition is

{

url: {

match: "/hello/:name/world"

}

}

then the URL must have three parts, the first part being hello and the third part world. For example, /hello/emil/world will match, while /hello/emil/meyer/world will not.

Constraint Match

A constraint match is similar to a parameterized match, but the parameters can carry constraints.

If the definition is

{

url: {

match: "/hello/:name/world",

constraint: {

name: "/[a-z]+/"

}

}

}

then the URL must have three parts, the first part being hello and the third part world. The second part must be all lowercase.

It is possible to use more then one constraint for the same URL part.

{

url: {

match: "/hello/:name|:id/world",

constraint: {

name: "/[a-z]+/", id: "/[0-9]+/"

}

}

}

Optional Match

An optional match is similar to a parameterized match, but the last parameter is optional.

If the definition is

{

url: {

match: "/hello/:name?",

constraint: {

name: "/[a-z]+/"

}

}

}

then the URL /hello and /hello/emil will match.

If the definitions are

{ url: { match: "/hello/world" } }

{ url: { match: "/hello/:name", constraint: { name: "/[a-z]+/" } } }

{ url: { match: "/hello/*" } }

then the URL /hello/world will be matched by the first route, because it is the most specific. The URL /hello/you will be matched by the second route, because it is more specific than the prefix match.

Method Restriction

You can restrict the match to specific HTTP methods.

If the definition is

{

url: {

match: "/hello/world",

methods: [ "post", "put" ]

}

}

then only HTTP POST and PUT requests will match. Calling with a different HTTP method will result in an HTTP 501 error.

Please note that if url is defined as a simple string, then only the HTTP methods GET and HEAD will be allowed, an all other methods will be disabled:

{

url: "/hello/world"

}

More on Matching

Remember that the more specific match wins.

- A match without parameter or wildcard is more specific than a match with parameters or wildcard.

- A match with parameter is more specific than a match with a wildcard.

- If there is more than one parameter, specificity is applied from left to right.

Consider the following definitions

(1) { url: { match: "/hello/world" } }

(2) { url: { match: "/hello/:name", constraint: { name: "/[a-z]+/" } } }

(3) { url: { match: "/:something/world" } }

(4) { url: { match: "/hello/*" } }

Then

- /hello/world is match by (1)

- /hello/emil is match by (2)

- /your/world is match by (3)

- /hello/you is match by (4)

You can write the following document into the _routing collection to test the above examples.

{

routes: [

{ url: { match: "/hello/world" }, content: "route 1" },

{ url: { match: "/hello/:name|:id", constraint: { name: "/[a-z]+/", id: "/[0-9]+/" } }, content: "route 2" },

{ url: { match: "/:something/world" }, content: "route 3" },

{ url: { match: "/hello/*" }, content: "route 4" },

]

}

A Hello World Example for JSON

If you change the example slightly, then a JSON object will be delivered.

arangosh> db._routing.save({

........>  url: "/hello/json",

........>  content: {

........>  contentType: "application/json",

........>    body: '{"hello" : "world"}'

........>  }

........> });

arangosh> require("internal").reloadRouting()

show execution results

Again check with your browser or cURL http://localhost:8529/hello/json

Depending on your browser and installed add-ons you will either see the JSON object or a download dialog. If your browser wants to open an external application to display the JSON object, you can change the contentType to "text/plain" for the example. This makes it easier to check the example using a browser. Or use curl to access the server.

shell> curl --dump - http://localhost:8529/hello/json

HTTP/1.1 200 OK

content-type: application/json

{

"hello" : "world"

}

Delivering Content

There are a lot of different ways on how to deliver content. We have already seen the simplest one, where static content is delivered. The fun, however, starts when delivering dynamic content.

Static Content

You can specify a body and a content-type.

arangosh> db._routing.save({

........>  url: "/hello/contentType",

........>  content: {

........>    contentType: "text/html",

........>    body: "<html><body>Hello World</body></html>"

........>  }

........> });

arangosh> require("internal").reloadRouting()

show execution results

shell> curl --dump - http://localhost:8529/hello/contentType

HTTP/1.1 200 OK

content-type: text/html

"Hello World"

If the content type is text/plain then you can use the short-cut

{

content: "Hello World"

}

A Simple Action

The simplest dynamic action is:

{

action: {

do: "org/arangodb/actions/echoRequest"

}

}

It is not advisable to store functions directly in the routing table. It is better to call functions defined in modules. In the above example the function can be accessed from JavaScript as:

require("org/arangodb/actions").echoRequest

The function echoRequest is pre-defined. It takes the request objects and echos it in the response.

The signature of such a function must be

function (req, res, options, next)

Examples

arangosh> db._routing.save({

........>    url: "/hello/echo",

........>    action: {

........>    do: "org/arangodb/actions/echoRequest"

........>  }

........> });

show execution results

Reload the routing and check http:// 127.0.0.1:8529/hello/echo

You should see something like

arangosh> arango.GET("/hello/echo")

show execution results

The request might contain path, prefix, suffix, and urlParameters attributes. path is the complete path as supplied by the user and always available. If a prefix was matched, then this prefix is stored in the attribute prefix and the remaining URL parts are stored as an array in suffix. If one or more parameters were matched, then the parameter values are stored in urlParameters.

For example, if the url description is

{

url: {

match: "/hello/:name/:action"

}

}

and you request the path /hello/emil/jump, then the request object will contain the following attribute

urlParameters: {

name: "emil",

action: "jump"

}

Action Controller

As an alternative to the simple action, you can use controllers. A controller is a module, defines the function get, put, post, delete, head, patch. If a request of the corresponding type is matched, the function will be called.

Examples

arangosh> db._routing.save({

........>  url: "/hello/echo",

........>  action: {

........>    controller: "org/arangodb/actions/echoController"

........>  }

........> });

show execution results

Reload the routing and check http:// 127.0.0.1:8529/hello/echo:

arangosh> arango.GET("/hello/echo")

show execution results

Prefix Action Controller

The controller is selected when the definition is read. There is a more flexible, but slower and maybe insecure variant, the prefix controller.

Assume that the url is a prefix match

{

url: {

match: /hello/*"

}

}

You can use

{

action: {

prefixController: "org/arangodb/actions"

}

}

to define a prefix controller. If the URL /hello/echoController is given, then the module org/arangodb/actions/echoController is used.

If you use a prefix controller, you should make certain that no unwanted actions are available under the prefix.

The definition

{

action: "org/arangodb/actions"

}

is a short-cut for a prefix controller definition.

Function Action

You can also store a function directly in the routing table.

Examples

arangosh> db._routing.save({

........>  url: "/hello/echo",

........>  action: {

........>    callback: "function(req,res) {res.statusCode=200; res.body='Hello'}"

........>  }

........> });

show execution results

arangosh> arango.GET("hello/echo")

Hello

arangosh> db._query("FOR route IN _routing FILTER route.url == '/hello/echo' REMOVE route in _routing")

[object ArangoQueryCursor, count: 0, hasMore: false]

arangosh> require("internal").reloadRouting()

Requests and Responses

The controller must define handler functions which take a request object and fill the response object.

A very simple example is the function echoRequest defined in the module org/arangodb/actions.

function (req, res, options, next) {

var result;

result = { request: req, options: options };

res.responseCode = exports.HTTP_OK;

res.contentType = "application/json";

res.body = JSON.stringify(result);

}

Install it via:

arangosh> db._routing.save({

........>  url: "/echo",

........>  action: {

........>    do: "org/arangodb/actions/echoRequest"

........>  }

........> })

show execution results

Reload the routing and check http:// 127.0.0.1:8529/hello/echo

You should see something like

arangosh> arango.GET("/hello/echo")

arangosh> db._query("FOR route IN _routing FILTER route.url == '/hello/echo' REMOVE route in _routing")

arangosh> require("internal").reloadRouting()

show execution results

You may also pass options to the called function:

arangosh> db._routing.save({

........>  url: "/echo",

........>  action: {

........>    do: "org/arangodb/actions/echoRequest",

........>    options: {

........>      "Hello": "World"

........>    }

........>  }

........> });

show execution results

You now see the options in the result:

arangosh> arango.GET("/echo")

arangosh> db._query("FOR route IN _routing FILTER route.url == '/echo' REMOVE route in _routing")

arangosh> require("internal").reloadRouting()

show execution results

Modifying Request and Response

As we've seen in the previous examples, actions get called with the request and response objects (named req and res in the examples) passed as parameters to their handler functions.

The req object contains the incoming HTTP request, which might or might not have been modified by a previous action (if actions were chained).

A handler can modify the request object in place if desired. This might be useful when writing middleware (see below) that is used to intercept incoming requests, modify them and pass them to the actual handlers.

While modifying the request object might not be that relevant for non-middleware actions, modifying the response object definitely is. Modifying the response object is an action's only way to return data to the caller of the action.

We've already seen how to set the HTTP status code, the content type, and the result body. The res object has the following properties for these:

- contentType: MIME type of the body as defined in the HTTP standard (e.g. text/html, text/plain, application/json, ...)

- responsecode: the HTTP status code of the response as defined in the HTTP standard. Common values for actions that succeed are 200 or 201. Please refer to the HTTP standard for more information.

- body: the actual response data

To set or modify arbitrary headers of the response object, the headers property can be used. For example, to add a user-defined header to the response, the following code will do:

res.headers = res.headers || { }; // headers might or might not be present

res.headers['X-Test'] = 'someValue'; // set header X-Test to "someValue"

This will set the additional HTTP header X-Test to value someValue. Other headers can be set as well. Note that ArangoDB might change the case of the header names to lower case when assembling the overall response that is sent to the caller.

It is not necessary to explicitly set a Content-Length header for the response as ArangoDB will calculate the content length automatically and add this header itself. ArangoDB might also add a Connection header itself to handle HTTP keep-alive.

ArangoDB also supports automatic transformation of the body data to another format. Currently, the only supported transformations are base64-encoding and base64-decoding. Using the transformations, an action can create a base64 encoded body and still let ArangoDB send the non-encoded version, for example:

res.body = 'VGhpcyBpcyBhIHRlc3Q=';

res.transformations = res.transformations || [ ]; // initialize

res.transformations.push('base64decode'); // will base64 decode the response body

When ArangoDB processes the response, it will base64-decode what's in res.body and set the HTTP header Content-Encoding: binary. The opposite can be achieved with the base64encode transformation: ArangoDB will then automatically base64-encode the body and set a Content-Encoding: base64 HTTP header.

Writing dynamic action handlers

To write your own dynamic action handlers, you must put them into modules.

Modules are a means of organizing action handlers and making them loadable under specific names.

To start, we'll define a simple action handler in a module /ownTest:

arangosh> db._modules.save({

........>  path: "/db:/ownTest",

........>  content:

........>     "exports.do = function(req, res, options, next) {"+

........>     "  res.body = 'test';" +

........>     "  res.responseCode = 200;" +

........>     "  res.contentType = 'text/plain';" +

........>     "};"

........> });

show execution results

This does nothing but register a do action handler in a module /ownTest. The action handler is not yet callable, but must be mapped to a route first. To map the action to the route /ourtest, execute the following command:

arangosh> db._routing.save({

........>  url: "/ourtest",

........>  action: {

........>    controller: "db://ownTest"

........>  }

........> });

arangosh> require("internal").reloadRouting()

show execution results

Now use the browser or cURL and access http://localhost:8529/ourtest :

shell> curl --dump - http://localhost:8529/ourtest

HTTP/1.1 200 OK

content-type: text/plain

"test"

You will see that the module's do function has been executed.

A Word about Caching

Sometimes it might seem that your change do not take effect. In this case the culprit could be the routing caches:

The routing cache stores the routing information computed from the _routing collection. Whenever you change this collection manually, you need to call

arangosh> require("internal").reloadRouting()

in order to rebuild the cache.

Advanced Usages

For detailed information see the reference manual.

Redirects

Use the following for a permanent redirect:

arangosh> db._routing.save({

........>  url: "/redirectMe",

........>  action: {

........>    do: "org/arangodb/actions/redirectRequest",

........>    options: {

........>      permanently: true,

........>      destination: "/somewhere.else/"

........>    }

........>  }

........> });

arangosh> require("internal").reloadRouting()

show execution results

shell> curl --dump - http://localhost:8529/redirectMe

HTTP/1.1 301 Moved Permanently

content-type: text/html

location: /somewhere.else/

"<html><head><title>Moved</title></head><body><h1>Moved</h1><p>This page has moved to <a href=\"/somewhere.else/\">/somewhere.else/</a>.</p></body></html>"

Routing Bundles

Instead of adding all routes for package separately, you can specify a bundle:

arangosh> db._routing.save({

........>  routes: [

........>    {

........>      url: "/url1",

........>      content: "route 1"

........>    },

........>    {

........>      url: "/url2",

........>      content: "route 2"

........>    },

........>    {

........>      url: "/url3",

........>      content: "route 3"

........>    }

........>  ]

........> });

arangosh> require("internal").reloadRouting()

show execution results

shell> curl --dump - http://localhost:8529/url2

HTTP/1.1 200 OK

content-type: text/plain

"route 2"

shell> curl --dump - http://localhost:8529/url3

HTTP/1.1 200 OK

content-type: text/plain

"route 3"

The advantage is, that you can put all your routes into one document and use a common prefix.

arangosh> db._routing.save({

........>  urlPrefix: "/test",

........>  routes: [

........>    {

........>      url: "/url1",

........>      content: "route 1"

........>    },

........>    {

........>      url: "/url2",

........>      content: "route 2"

........>    },

........>    {

........>      url: "/url3",

........>      content: "route 3"

........>    }

........>  ]

........> });

arangosh> require("internal").reloadRouting()

show execution results

will define the URL /test/url1, /test/url2, and /test/url3:

shell> curl --dump - http://localhost:8529/test/url1

HTTP/1.1 200 OK

content-type: text/plain

"route 1"

shell> curl --dump - http://localhost:8529/test/url2

HTTP/1.1 200 OK

content-type: text/plain

"route 2"

shell> curl --dump - http://localhost:8529/test/url3

HTTP/1.1 200 OK

content-type: text/plain

"route 3"

Writing Middleware

Assume, you want to log every request in your namespace to the console. (if ArangoDB is running as a daemon, this will end up in the logfile). In this case you can easily define an action for the URL /subdirectory. This action simply logs the requests, calls the next in line, and logs the response:

arangosh> db._modules.save({

........>  path: "/db:/OwnMiddlewareTest",

........>  content:

........>     "exports.logRequest = function (req, res, options, next) {" +

........>     "    console = require('console'); " +

........>     "    console.log('received request: %s', JSON.stringify(req));" +

........>     "    next();" +

........>     "    console.log('produced response: %s', JSON.stringify(res));" +

........>     "};"

........> });

show execution results

This function will now be available as db://OwnMiddlewareTest/logRequest. You need to tell ArangoDB that it is should use a prefix match and that the shortest match should win in this case:

arangosh> db._routing.save({

........>  middleware: [

........>    {

........>      url: {

........>        match: "/subdirectory/*"

........>      },

........>      action: {

........>        do: "db://OwnMiddlewareTest/logRequest"

........>      }

........>    }

........>  ]

........> });

show execution results

When you call next() in that action, the next specific routing will be used for the original URL. Even if you modify the URL in the request object req, this will not cause the next() to jump to the routing defined for this next URL. If proceeds occurring the origin URL. However, if you use next(true), the routing will stop and request handling is started with the new URL. You must ensure that next(true) is never called without modifying the URL in the request object req. Otherwise an endless loop will occur.

Now we add some other simple routings to test all this:

arangosh> db._routing.save({

........>    url: "/subdirectory/ourtest/1",

........>    action: {

........>      do: "org/arangodb/actions/echoRequest"

........>    }

........> });

arangosh> db._routing.save({

........>    url: "/subdirectory/ourtest/2",

........>    action: {

........>      do: "org/arangodb/actions/echoRequest"

........>    }

........> });

arangosh> db._routing.save({

........>    url: "/subdirectory/ourtest/3",

........>    action: {

........>      do: "org/arangodb/actions/echoRequest"

........>    }

........> });

arangosh> require("internal").reloadRouting()

show execution results

Then we send some curl requests to these sample routes:

shell> curl --dump - http://localhost:8529/subdirectory/ourtest/1

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

and the console (and / or the logfile) will show requests and replies. Note that logging doesn't warrant the sequence in which these lines will appear.

Application Deployment

Using single routes or bundles can be become a bit messy in large applications. Kaerus has written a deployment tool in node.js.

Note that there is also Foxx for building applications with ArangoDB.

Common Pitfalls when using Actions

Caching

If you made any changes to the routing but the changes does not have any effect when calling the modified actions URL, you might have been hit by some caching issues.

After any modification to the routing or actions, it is thus recommended to make the changes "live" by calling the following functions from within arangosh:

You might also be affected by client-side caching. Browsers tend to cache content and also redirection URLs. You might need to clear or disable the browser cache in some cases to see your changes in effect.

Data types

When processing the request data in an action, please be aware that the data type of all query parameters is string. This is because the whole URL is a string and when the individual parts are extracted, they will also be strings.

For example, when calling the URL http:// localhost:8529/hello/world?value=5

the parameter value will have a value of (string) 5, not (number) 5. This might be troublesome if you use JavaScript's === operator when checking request parameter values.

The same problem occurs with incoming HTTP headers. When sending the following header from a client to ArangoDB

X-My-Value: 5

then the header X-My-Value will have a value of (string) 5 and not (number) 5.

404 Not Found

If you defined a URL in the routing and the URL is accessible fine via HTTP GET but returns an HTTP 501 (not implemented) for other HTTP methods such as POST, PUT or DELETE, then you might have been hit by some defaults.

By default, URLs defined like this (simple string url attribute) are accessible via HTTP GET and HEAD only. To make such URLs accessible via other HTTP methods, extend the URL definition with the methods attribute.

For example, this definition only allows access via GET and HEAD:

{

url: "/hello/world"

}

whereas this definition allows HTTP GET, POST, and PUT:

arangosh> db._routing.save({

........>    url: {

........>      match: "/hello/world",

........>      methods: [ "get", "post", "put" ]

........>    },

........>    action: {

........>      do: "org/arangodb/actions/echoRequest"

........>    }

........> });

arangosh> require("internal").reloadRouting()

show execution results

shell> curl --dump - http://localhost:8529/hello/world

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

The former definition (defining url as an object with a match attribute) will result in the URL being accessible via all supported HTTP methods (e.g. GET, POST, PUT, DELETE, ...), whereas the latter definition (providing a string url attribute) will result in the URL being accessible via HTTP GET and HTTP HEAD only, with all other HTTP methods being disabled. Calling a URL with an unsupported or disabled HTTP method will result in an HTTP 404 error.

Introduction to Replication

How the replication works

Starting with ArangoDB 1.4, ArangoDB comes with optional asynchronous master-slave replication. Replication is configured on a per-database level, meaning that different databases in the same ArangoDB instance can have different replication settings. Replication must be turned on explicitly before it becomes active for a database.

In a typical master-slave replication setup, clients direct all their write operations for a specific database to the master. The master database is the only place to connect to when making any insertions/updates/deletions.

The master database will log all write operations in its write-ahead log. Any number of slaves can then connect to the master database and fetch data from the master database's write-ahead log. The slaves then can apply all the events from the log in the same order locally. After that, they will have the same state of data as the master database.

Transactions are honored in replication, i.e. transactional write operations will become visible on slaves atomically.

As all write operations will be logged to a master database's write-ahead log, the replication in ArangoDB currently cannot be used for write-scaling. The main purposes of the replication in current ArangoDB are to provide read-scalability and "hot backups" of specific databases.

It is possible to connect multiple slave databases to the same master database. Slave databases should be used as read-only instances, and no user-initiated write operations should be carried out on them. Otherwise data conflicts may occur that cannot be solved automatically, and that will make the replication stop. Master-master (or multi-master) replication is not currently supported in ArangoDB.

The replication in ArangoDB is asynchronous, meaning that slaves will pull changes from the master database. Slaves need to know to which master database they should connect to, but a master database is not aware of the slaves that replicate from it. When the network connection between the master database and a slave goes down, write operations on the master can continue normally. When the network is up again, slaves can reconnect to the master database and transfer the remaining changes. This will happen automatically provided slaves are configured appropriately.

Replication lag

In this setup, write operations are applied first in the master database, and applied in the slave database(s) afterwards.

For example, let's assume a write operation is executed in the master database at point in time t0. To make a slave database apply the same operation, it must first fetch the write operation's data from master database's write-ahead log, then parse it and apply it locally. This will happen at some point in time after t0, let's say t1.

The difference between t1 and t0 is called the replication lag, and it is unavoidable in asynchronous replication. The amount of replication lag depends on many factors, a few of which are:

- the network capacity between the slaves and the master

- the load of the master and the slaves

- the frequency in which slaves poll the master for updates

Between t0 and t1, the state of data on the master is newer than the state of data on the slave(s). At point in time t1, the state of data on the master and slave(s) is consistent again (provided no new data modifications happened on the master in between). Thus, the replication will lead to an eventually consistent state of data.

Replication configuration

The replication is turned off by default. In order to create a master-slave setup, the so-called replication applier needs to be enabled on the slave databases. Since ArangoDB 2.2, the master database does not need any special configuration, as it will automatically log all data modification operations into its write-ahead log. The write-ahead log is then used in replication. Versions prior to ArangoDB 2.2 required the so-called replication logger to be activated on the master, too. This requirement is gone since ArangoDB 2.2 and the replication logger does not have any purpose in ArangoDB 2.2 and higher. The replication logger object is kept for compatibility with previous versions only.

Replication is configured on a per-database level. If multiple database are to be replicated, the replication must be set up individually per database.

The replication applier on the slave can be used to perform a one-time synchronization with the master (and then stop), or to perform an ongoing replication of changes. To resume replication on slave restart, the autoStart attribute of the replication applier must be set to true.

Components

The replication architecture in ArangoDB before version 2.2 consisted of two main components, which could be used together or in isolation: the replication logger and the replication applier. Since ArangoDB 2.2, the replication logger has no special purpose anymore and is available for downwards-compatibility only.

The replication applier can be administered via the command line or a REST API (see HTTP Interface for Replication).

As replication is configured on a per-database level and there can be multiple databases inside one ArangoDB instance, there can be multiple replication appliers in one ArangoDB instance.

Replication Logger

Purpose

The purpose of the replication logger in ArangoDB before version 2.2 was to log all changes that modify the state of data in a specific database on a master server. This included document insertions, updates, and deletions as well as creating, dropping, renaming and changing collections and indexes.

When the replication logger was used, it logged all these write operations for the database in its own event log, which was a system collection named _replication.

Reading the events sequentially from the _replication collection provides a list of all write operations carried out in the master database. Replication clients could then request these events from the logger and apply them on their own.

Starting with ArangoDB 2.2, the _replication system collection is not used anymore. Instead, the server will write all data-modification operations into its write-ahead log. The write-ahead log can be queried by clients, so the need for an extra event log is gone.

Checking the state

Starting with version 2.2, ArangoDB will log all data-modification operations in its write-ahead log automatically. There is no need to explicitly start or configure the replication logger on the master.

To query the current state of the logger, use the state command:

require("org/arangodb/replication").logger.state();

The result might look like this:

{

"state" : {

"running" : true,

"lastLogTick" : "133322013",

"totalEvents" : 16,

"time" : "2014-07-06T12:58:11Z"

},

"server" : {

"version" : "2.2.0-devel",

"serverId" : "40897075811372"

},

"clients" : {

}

}

In old versions of ArangoDB, the running attribute indicated whether the logger was currently enabled and logged any events. In ArangoDB 2.2 and higher, this attribute will always have a value of true, as the replication logger cannot be turned off.

The totalEvents attribute indicates how many log events have been logged since the start of the ArangoDB server. Finally, the lastLogTick value indicates the id of the last operation that was written to the server's write-ahead log. It can be used to determine whether new operations were logged, and is also used by the replication applier for incremental fetching of data.

Note: The replication logger state can also be queried via the HTTP API.

To query which data ranges are still available for replication clients to fetch, the logger provides the firstTick and tickRanges functions:

require("org/arangodb/replication").logger.firstTick();

This will return the minimum tick value that the server can provide to replication clients via its replication APIS. The tickRanges function returns the minimum and maximum tick values per logfile:

require("org/arangodb/replication").logger.tickRanges();

Configuration

Since ArangoDB 2.2, no special configuration is necessary for the replication logger. All operations are written to a server's write-ahead log, and the write-ahead log is used for replication, too.

Replication Applier

Purpose

The purpose of the replication applier is to read data from a master database's event log, and apply them locally. The applier will check the master database for new operations periodically. It will perform an incremental synchronization, i.e. only asking the master for operations that occurred after the last synchronization.

The replication applier does not get notified by the master database when there are "new" operations available, but instead uses the pull principle. It might thus take some time (the so-called replication lag) before an operation from the master database gets shipped to and applied in a slave database.

The replication applier of a database is run in a separate thread. It may encounter problems when an operation from the master cannot be applied safely, or when the connection to the master database goes down (network outage, master database is down or unavailable etc.). In this case, the database's replication applier thread might terminate itself. It is then up to the administrator to fix the problem and restart the database's replication applier.

If the replication applier cannot connect to the master database, or the communication fails at some point during the synchronization, the replication applier will try to reconnect to the master database. It will give up reconnecting only after a configurable amount of connection attempts.

The replication applier state is queryable at any time by using the state command of the applier. This will return the state of the applier of the current database:

require("org/arangodb/replication").applier.state();

The result might look like this:

{

"state" : {

"running" : true,

"lastAppliedContinuousTick" : "152786205",

"lastProcessedContinuousTick" : "152786205",

"lastAvailableContinuousTick" : "152786205",

"progress" : {

"time" : "2014-07-06T13:04:57Z",

"message" : "fetching master log from offset 152786205",

"failedConnects" : 0

},

"totalRequests" : 38,

"totalFailedConnects" : 0,

"totalEvents" : 1,

"lastError" : {

"errorNum" : 0

},

"time" : "2014-07-06T13:04:57Z"

},

"server" : {

"version" : "2.2.0-devel",

"serverId" : "210189384542896"

},

"endpoint" : "tcp://master.example.org:8529",

"database" : "_system"

}

The running attribute indicates whether the replication applier of the current database is currently running and polling the server at endpoint for new events.

The progress.failedConnects attribute shows how many failed connection attempts the replication applier currently has encountered in a row. In contrast, the totalFailedConnects attribute indicates how many failed connection attempts the applier has made in total. The totalRequests attribute shows how many requests the applier has sent to the master database in total. The totalEvents attribute shows how many log events the applier has read from the master.

The progress.message sub-attribute provides a brief hint of what the applier currently does (if it is running). The lastError attribute also has an optional errorMessage sub-attribute, showing the latest error message. The errorNum sub-attribute of the lastError attribute can be used by clients to programmatically check for errors. It should be 0 if there is no error, and it should be non-zero if the applier terminated itself due to a problem.

Here is an example of the state after the replication applier terminated itself due to (repeated) connection problems:

{

"state" : {

"running" : false,

"progress" : {

"time" : "2014-07-06T13:14:37Z",

"message" : "applier stopped",

"failedConnects" : 6

},

"totalRequests" : 79,

"totalFailedConnects" : 11,

"totalEvents" : 0,

"lastError" : {

"time" : "2014-07-06T13:09:41Z",

"errorMessage" : "could not connect to master at tcp://master.example.org:8529: Could not connect to 'tcp:/...",

"errorNum" : 1400

},

...

}

}

Note: the state of a database's replication applier is queryable via the HTTP API, too. Please refer to HTTP Interface for Replication for more details.

All-in-one setup

To copy the initial data from the slave to the master and start the continuous replication, there is an all-in-one command setupReplication:

require("org/arangodb/replication").setupReplication(configuration);

The following example demonstrates how to use the command for setting up replication for the _system database. Note that it should be run on the slave and not the master:

db._useDatabase("_system");

require("org/arangodb/replication").setupReplication({

endpoint: "tcp://master.domain.org:8529",

username: "myuser",

password: "mypasswd",

verbose: false,

includeSystem: false,

incremental: true,

autoResync: true

});

The command will return when the initial synchronization is finished and the continuous replication is started, or in case the initial synchronization has failed.

If the initial synchronization is successful, the command will store the given configuration on the slave. It also configures the continuous replication to start automatically if the slave is restarted, i.e. autoStart is set to true.

If the command is run while the slave's replication applier is already running, it will first stop the running applier, drop its configuration and do a resynchronization of data with the master. It will then use the provided configration, overwriting any previously existing replication configuration on the slave.

Starting and Stopping

To manually start and stop the applier in the current database, the start and stop commands can be used like this:

require("org/arangodb/replication").applier.start(<tick>);

require("org/arangodb/replication").applier.stop();

Note: Starting a replication applier without setting up an initial configuration will fail. The replication applier will look for its configuration in a file named REPLICATION-APPLIER-CONFIG in the current database's directory. If the file is not present, ArangoDB will use some default configuration, but it cannot guess the endpoint (the address of the master database) the applier should connect to. Thus starting the applier without configuration will fail.

Note that at the first time you start the applier, you should pass the value returned in the lastLogTick attribute of the initial sync operation.

Note: Starting a database's replication applier via the start command will not necessarily start the applier on the next and following ArangoDB server restarts. Additionally, stopping a database's replication applier manually will not necessarily prevent the applier from being started again on the next server start. All of this is configurable separately (hang on reading).

Note: when stopping and restarting the replication applier of database, it will resume where it last stopped. This is sensible because replication log events should be applied incrementally. If the replication applier of a database has never been started before, it needs some tick value from the master's log from which to start fetching events.

There is one caveat to consider when stopping a replication on the slave: if there are still ongoing replicated transactions that are neither committed or aborted, stopping the replication applier will cause these operations to be lost for the slave. If these transactions commit on the master later and the replication is resumed, the slave will not be able to commit these transactions, too. Thus stopping the replication applier on the slave manually should only be done if there is certainty that there are no ongoing transactions on the master.

Configuration

To configure the replication applier of a specific database, use the properties command. Using it without any arguments will return the applier's current configuration:

require("org/arangodb/replication").applier.properties();

The result might look like this:

{

"requestTimeout" : 600,

"connectTimeout" : 10,

"ignoreErrors" : 0,

"maxConnectRetries" : 10,

"chunkSize" : 0,

"autoStart" : false,

"adaptivePolling" : true,

"includeSystem" : true,

"requireFromPresent" : false,

"autoResync" : false,

"autoResyncRetries" : 2,

"verbose" : false

}

Note: There is no endpoint attribute configured yet. The endpoint attribute is required for the replication applier to be startable. You may also want to configure a username and password for the connection via the username and password attributes.

require("org/arangodb/replication").applier.properties({

endpoint: "tcp://master.domain.org:8529",

username:  "root",

password: "secret",

verbose: false

});

This will re-configure the replication applier for the current database. The configuration will be used from the next start of the replication applier. The replication applier cannot be re-configured while it is running. It must be stopped first to be re-configured.

To make the replication applier of the current database start automatically when the ArangoDB server starts, use the autoStart attribute.

Setting the adaptivePolling attribute to true will make the replication applier poll the master database for changes with a variable frequency. The replication applier will then lower the frequency when the master is idle, and increase it when the master can provide new events). Otherwise the replication applier will poll the master database for changes with a constant frequency.

The idleMinWaitTime attribute controls the minimum wait time (in seconds) that the replication applier will intentionally idle before fetching more log data from the master in case the master has already sent all its log data. This wait time can be used to control the frequency with which the replication applier sends HTTP log fetch requests to the master in case there is no write activity on the master.

The idleMaxWaitTime attribute controls the maximum wait time (in seconds) that the replication applier will intentionally idle before fetching more log data from the master in case the master has already sent all its log data and there have been previous log fetch attempts that resulted in no more log data. This wait time can be used to control the maximum frequency with which the replication applier sends HTTP log fetch requests to the master in case there is no write activity on the master for longer periods. Note that this configuration value will only be used if the option adaptivePolling is set to true.

To set a timeout for connection and following request attempts, use the connectTimeout and requestTimeout values. The maxConnectRetries attribute configures after how many failed connection attempts in a row the replication applier will give up and turn itself off. You may want to set this to a high value so that temporary network outages do not lead to the replication applier stopping itself. The connectRetryWaitTime attribute configures how long the replication applier will wait before retrying the connection to the master in case of connection problems.

The chunkSize attribute can be used to control the approximate maximum size of a master's response (in bytes). Setting it to a low value may make the master respond faster (less data is assembled before the master sends the response), but may require more request-response roundtrips. Set it to 0 to use ArangoDB's built-in default value.

The includeSystem attribute controls whether changes to system collections (such as _graphs or _users) should be applied. If set to true, changes in these collections will be replicated, otherwise, they will not be replicated. It is often not necessary to replicate data from system collections, especially because it may lead to confusion on the slave because the slave needs to have its own system collections in order to start and keep operational.

The requireFromPresent attribute controls whether the applier will start synchronizing in case it detects that the master cannot provide data for the initial tick value provided by the slave. This may be the case if the master does not have a big enough backlog of historic WAL logfiles, and when the replication is re-started after a longer pause. When requireFromPresent is set to true, then the replication applier will check at start whether the start tick from which it starts or resumes replication is still present on the master. If not, then there would be data loss. If requireFromPresent is true, the replication applier will abort with an appropriate error message. If set to false, then the replication applier will still start, and ignore the data loss.

The autoResync option can be used in conjunction with the requireFromPresent option as follows: when both requireFromPresent and autoResync are set to true and the master cannot provide the log data the slave requests, the replication applier will stop as usual. But due to the fact that autoResync is set to true, the slave will automatically trigger a full resync of all data with the master. After that, the replication applier will go into continuous replication mode again. Additionally, setting autoResync to true will trigger a full re-synchronization of data when the continuous replication is started and detects that there is no start tick value.

Automatic re-synchronization may transfer a lot of data from the master to the slave and can be expensive. It is therefore turned off by default. When turned off, the slave will never perform an automatic re-synchronization with the master.

The autoResyncRetries option can be used to control the number of resynchronization retries that will be performed in a row when automatic resynchronization is enabled and kicks in. Setting this to 0 will effectively disable autoResync. Setting it to some other value will limit the number of retries that are performed. This helps preventing endless retries in case resynchronizations always fail.

The verbose attribute controls the verbosity of the replication logger. Setting it to  true  will make the replication applier write a line to the log for every operation it performs. This should only be used for diagnosing replication problems.

The following example will set most of the discussed properties for the current database's applier:

require("org/arangodb/replication").applier.properties({

endpoint: "tcp://master.domain.org:8529",

username: "root",

password: "secret",

adaptivePolling: true,

connectTimeout: 15,

maxConnectRetries: 100,

chunkSize: 262144,

autoStart: true,

includeSystem: true,

autoResync: true,

autoResyncRetries: 2,

});

After the applier is now fully configured, it could theoretically be started. However, we may first need an initial synchronization of all collections and their data from the master before we start the replication applier.

The only safe method for doing a full synchronization (or re-synchronization) is thus to

- stop the replication applier on the slave (if currently running)

- perform an initial full sync with the master database

- note the master database's lastLogTick value and

- start the continuous replication applier on the slave using this tick value.

The initial synchronization for the current database is executed with the sync command:

require("org/arangodb/replication").sync({

endpoint: "tcp://master.domain.org:8529",

username: "root",

password: "secret,

includeSystem: true

});

The includeSystem option controls whether data from system collections (such as _graphs and _users) shall be synchronized.

The initial synchronization can optionally be configured to include or exclude specific collections using the restrictType and restrictCollection parameters.

The following command only synchronizes collection foo and bar:

require("org/arangodb/replication").sync({

endpoint: "tcp://master.domain.org:8529",

username: "root",

password: "secret,

restrictType: "include",

restrictCollections: [ "foo", "bar" ]

});

Using a restrictType of exclude, all collections but the specified will be synchronized.

Warning: sync will do a full synchronization of the collections in the current database with collections present in the master database. Any local instances of the collections and all their data are removed! Only execute this command if you are sure you want to remove the local data!

As sync does a full synchronization, it might take a while to execute. When sync completes successfully, it returns an array of collections it has synchronized in its collections attribute. It will also return the master database's last log tick value at the time the sync was started on the master. The tick value is contained in the lastLogTick attribute of the sync command:

{

"lastLogTick" : "231848833079705",

"collections" : [ ... ]

}

Now you can start the continuous synchronization for the current database on the slave with the command

require("org/arangodb/replication").applier.start("231848833079705");

Note: The tick values should be treated as strings. Using numeric data types for tick values is unsafe because they might exceed the 32 bit value and the IEEE754 double accuracy ranges.

Example Setup

Setting up a working master-slave replication requires two ArangoDB instances:

- master: this is the instance that all data-modification operations should be directed to

- slave: on this instance, we'll start a replication applier, and this will fetch data from the master database's write-ahead log and apply its operations locally

For the following example setup, we'll use the instance tcp://master.domain.org:8529 as the master, and the instance tcp://slave.domain.org:8530 as a slave.

The goal is to have all data from the database _system on master tcp://master.domain.org:8529 be replicated to the database _system on the slave tcp://slave.domain.org:8530.

On the master, nothing special needs to be done, as all write operations will automatically be logged in the master's write-ahead log (WAL).

All-in-one setup

To make the replication copy the initial data from the master to the slave and start the continuous replication on the slave, there is an all-in-one command:

require("org/arangodb/replication").setupReplication(configuration);

The following example demonstrates how to use the command for setting up replication for the _system database. Note that it should be run on the slave and not the master:

db._useDatabase("_system");

require("org/arangodb/replication").setupReplication({

endpoint: "tcp://master.domain.org:8529",

username: "myuser",

password: "mypasswd",

verbose: false,

includeSystem: false,

incremental: true,

autoResync: true

});

The command will return when the initial synchronization is finished and the continuous replication has been started, or in case the initial synchronization has failed.

If the initial synchronization is successful, the command will store the given configuration on the slave. It also configures the continuous replication to start automatically if the slave is restarted, i.e. autoStart is set to true.

If the command is run while the slave's replication applier is already running, it will first stop the running applier, drop its configuration and do a resynchronization of data with the master. It will then use the provided configration, overwriting any previously existing replication configuration on the slave.

Initial synchronization

The initial synchronization and continuous replication applier can also be started separately. To start replication on the slave, make sure there currently is no replication applier running.

The following commands stop a running applier in the slave's _system database:

db._useDatabase("_system");

require("org/arangodb/replication").applier.stop();

The stop operation will terminate any replication activity in the _system database on the slave.

After that, the initial synchronization can be run. It will copy the collections from the master to the slave, overwriting existing data. To run the initial synchronization, execute the following commands on the slave:

db._useDatabase("_system");

require("org/arangodb/replication").sync({

endpoint: "tcp://master.domain.org:8529",

username: "myuser",

password: "mypasswd",

verbose: false

});

Username and password only need to be specified when the master requires authentication. To check what the synchronization is currently doing, supply set the verbose option to true. If set, the synchronization will create log messages with the current synchronization status.

Warning: The sync command will replace data in the slave database with data from the master database! Only execute these commands if you have verified you are on the correct server, in the correct database!

The sync operation will return an attribute named lastLogTick which we'll need to note. The last log tick will be used as the starting point for subsequent replication activity. Let's assume we got the following last log tick:

{

"lastLogTick" : "40694126",

...

}

Initial synchronization from the ArangoShell

The initial synchronization via the sync command may take a long time to complete. The shell will block until the slave has completed the initial synchronization or until an error occurs. By default, the sync command in the ArangoShell will poll the slave for a status update every 10 seconds.

Optionally the sync command can be made non-blocking by setting its async option to true. In this case, the sync command will return instantly with an id string, and the initial synchronization will run detached on the master. To fetch the current status of the sync progress from the ArangoShell, the getSyncResult function can be used as follows:

db._useDatabase("_system");

var replication = require("org/arangodb/replication");

/* run command in async mode */

var id = replication.sync({

endpoint: "tcp://master.domain.org:8529",

username: "myuser",

password: "mypasswd",

async: true

});

/* now query the status of our operation */

print(replication.getSyncResult(id));

getSyncResult will return false as long as the synchronization is not complete, and return the synchronization result otherwise.

Continuous synchronization

When the initial synchronization is finished, the continuous replication applier can be started using the last log tick provided by the sync command. Before starting it, there is at least one configuration option to consider: replication on the slave will be running until the slave gets shut down. When the slave server gets restarted, replication will be turned off again. To change this, we first need to configure the slave's replication applier and set its autoStart attribute.

Here's the command to configure the replication applier with several options, including the autoStart attribute:

db._useDatabase("_system");

require("org/arangodb/replication").applier.properties({

endpoint: "tcp://master.domain.org:8529",

username: "myuser",

password: "mypasswd",

autoStart: true,

autoResync: true,

autoResyncRetries: 2,

adaptivePolling: true,

includeSystem: false,

requireFromPresent: false,

idleMinWaitTime: 0.5,

idleMaxWaitTime: 1.5,

verbose: false

});

An important consideration for replication is whether data from system collections (such as _graphs or _users) should be applied. The includeSystem option controls that. If set to true, changes in system collections will be replicated. Otherwise, they will not be replicated. It is often not necessary to replicate data from system collections, especially because it may lead to confusion on the slave because the slave needs to have its own system collections in order to start and keep operational.

The requireFromPresent attribute controls whether the applier will start synchronizing in case it detects that the master cannot provide data for the initial tick value provided by the slave. This may be the case if the master does not have a big enough backlog of historic WAL logfiles, and when the replication is re-started after a longer pause. When requireFromPresent is set to true, then the replication applier will check at start whether the start tick from which it starts or resumes replication is still present on the master. If not, then there would be data loss. If requireFromPresent is true, the replication applier will abort with an appropriate error message. If set to false, then the replication applier will still start, and ignore the data loss.

The autoResync option can be used in conjunction with the requireFromPresent option as follows: when both requireFromPresent and autoResync are set to true and the master cannot provide the log data the slave had requested, the replication applier will stop as usual. But due to the fact that autoResync is set to true, the slave will automatically trigger a full resync of all data with the master. After that, the replication applier will go into continuous replication mode again. Additionally, setting autoResync to true will trigger a full re-synchronization of data when the continuous replication is started and detects that there is no start tick value.

Note that automatic re-synchronization (autoResync option set to true) may transfer a lot of data from the master to the slave and can therefore be expensive. Still it's turned on here so there's less need for manual intervention.

The autoResyncRetries option can be used to control the number of resynchronization retries that will be performed in a row when automatic resynchronization is enabled and kicks in. Setting this to 0 will effectively disable autoResync. Setting it to some other value will limit the number of retries that are performed. This helps preventing endless retries in case resynchronizations always fail.

Now it's time to start the replication applier on the slave using the last log tick we got before:

db._useDatabase("_system");

require("org/arangodb/replication").applier.start("40694126");

This will replicate all operations happening in the master's system database and apply them on the slave, too.

After that, you should be able to monitor the state and progress of the replication applier by executing the state command on the slave server:

db._useDatabase("_system");

require("org/arangodb/replication").applier.state();

Please note that stopping the replication applier on the slave using the stop command should be avoided. The reason is that currently ongoing transactions (that have partly been replicated to the slave) will be need to be restarted after a restart of the replication applier. Stopping and restarting the replication applier on the slave should thus only be performed if there is certainty that the master is currently fully idle and all transactions have been replicated fully.

Note that while a slave has only partly executed a transaction from the master, it might keep a write lock on the collections involved in the transaction.

You may also want to check the master and slave states via the HTTP APIs (see HTTP Interface for Replication).

Syncing Collections

In order to synchronize data for a single collection from a master to a slave instance, there is the syncCollection function:

It will fetch all documents of the specified collection from the master database and store them in the local instance. After the synchronization, the collection data on the slave will be identical to the data on the master, provided no further data changes happen on the master. Any data changes that are performed on the master after the synchronization was started will not be captured by syncCollection, but need to be replicated using the regular replication applier mechanism.

For the following example setup, we'll use the instance tcp://master.domain.org:8529 as the master, and the instance tcp://slave.domain.org:8530 as a slave.

The goal is to have all data from the collection test in database _system on master tcp://master.domain.org:8529 be replicated to the collection test in database _system on the slave tcp://slave.domain.org:8530.

On the master, the collection test needs to be present in the _system database, with any data in it.

To transfer this collection to the slave, issue the following commands there:

db._useDatabase("_system");

require("org/arangodb/replication").syncCollection("test", {

endpoint: "tcp://master.domain.org:8529",

username: "myuser",

password: "mypasswd"

});

Warning: The syncCollection command will replace the collection's data in the slave database with data from the master database! Only execute these commands if you have verified you are on the correct server, in the correct database!

Setting the optional incremental attribute in the call to syncCollection will start an incremental transfer of data. This may be useful in case when the slave already has parts or almost all of the data in the collection and only the differences need to be synchronized. Note that to compute the differences the incremental transfer will build a sorted list of all document keys in the collection on both the slave and the master, which may still be expensive for huge collections in terms of memory usage and runtime. During building the list of keys the collection will be read-locked on the master.

The initialSyncMaxWaitTime attribute in the call to syncCollection controls how long the slave will wait for a master's response. This wait time can be used to control after what time the synchronization will give up and fail.

The syncCollection command may take a long time to complete if the collection is big. The shell will block until the slave has synchronized the entire collection from the master or until an error occurs. By default, the syncCollection command in the ArangoShell will poll for a status update every 10 seconds.

When syncCollection is called from the ArangoShell, the optional async attribute can be used to start the synchronization as a background process on the slave. If async is set to true, the call to syncCollection will return almost instantly with an id string. Using this id string, the status of the sync job on the slave can be queried using the getSyncResult function as follows:

db._useDatabase("_system");

var replication = require("org/arangodb/replication");

/* run command in async mode */

var id = replication.syncCollection("test", {

endpoint: "tcp://master.domain.org:8529",

username: "myuser",

password: "mypasswd",

async: true

});

/* now query the status of our operation */

print(replication.getSyncResult(id));

getSyncResult will return false as long as the synchronization is not complete, and return the synchronization result otherwise.

Replication Limitations

The replication in ArangoDB has a few limitations. Some of these limitations may be removed in later versions of ArangoDB:

- there is no feedback from the slaves to the master. If a slave cannot apply an event it got from the master, the master will have a different state of data. In this case, the replication applier on the slave will stop and report an error. Administrators can then either "fix" the problem or re-sync the data from the master to the slave and start the applier again.

- the replication is an asynchronous master-slave replication. There is currently no way to use it as a synchronous replication, or a multi-master replication.

- at the moment it is assumed that only the replication applier executes write operations on a slave. ArangoDB currently does not prevent users from carrying out their own write operations on slaves, though this might lead to undefined behavior and the replication applier stopping.

- when a replication slave asks a master for log events, the replication master will return all write operations for user-defined collections, but it will exclude write operations for certain system collections. The following collections are excluded intentionally from replication: _apps, _trx, _replication, _configuration, _cluster_kickstart_plans, _jobs, _queues, _sessions, _foxxlog and all statistics collections. Write operations for the following system collections can be queried from a master: _aqlfunctions, _graphs, _users.

- Foxx applications consist of database entries and application scripts in the file system. The file system parts of Foxx applications are not tracked anywhere and thus not replicated in current versions of ArangoDB. To replicate a Foxx application, it is required to copy the application to the remote server and install it there using the foxx-manager utility.

- master servers do not know which slaves are or will be connected to them. All servers in a replication setup are currently only loosely coupled. There currently is no way for a client to query which servers are present in a replication.

- failover must currently be handled by clients or client APIs.

- there currently is one replication applier per ArangoDB database. It is thus not possible to have a slave apply operations from multiple masters into the same target database.

- replication is set up on a per-database level. When using ArangoDB with multiple databases, replication must be configured individually for each database.

- the replication applier is single-threaded, but write operations on the master may be executed in parallel if they affect different collections. Thus the replication applier might not be able to catch up with a very powerful and loaded master.

- replication is only supported between the two ArangoDB servers running the same ArangoDB version. It is currently not possible to replicate between different ArangoDB versions.

- a replication applier cannot apply data from itself.

Replication Overhead

In ArangoDB before version 2.2, there was a so-called replication logger which needed to be activated on the master in order to allow replicating from it. If the logger was not activated, slaves could not replicate from the master. Running the replication logger caused extra write operations on the master, which made all data modification operations more expensive. Each data modification operation was first executed normally, and then was additionally written to the master's replication log. Turning on the replication logger may have reduce throughput on an ArangoDB master by some extent. If the replication feature was not required, the replication logger should have been turned off to avoid this reduction.

Since ArangoDB 2.2, a master will automatically write all data modification operations into its write-ahead log, which can also be used for replication. No separate replication log is written on the master anymore. This reduces the overhead of replication on the master when compared to previous versions of ArangoDB. In fact, a master server that is used in a replication setup will have the same throughput than a standalone server that is not used in replication.

Slaves that connect to an ArangoDB master will cause some work on the master as the master needs to process the incoming HTTP requests, return the requested data from its write-ahead and send the response.

In ArangoDB versions prior to 2.2, transactions were logged on the master as an uninterrupted sequence, restricting their maximal size considerably. While a transaction was written to the master's replication log, any other replication logging activity was blocked.

This is not the case since ArangoDB 2.2. Transactions are now written to the write-ahead log as the operations of the transactions occur. They may be interleaved with other operations.

Sharding

Sharding allows to use multiple machines to run a cluster of ArangoDB instances that together constitute a single database. This enables you to store much more data, since ArangoDB distributes the data automatically to the different servers. In many situations one can also reap a benefit in data throughput, again because the load can be distributed to multiple machines.

In a cluster there are essentially two types of processes: "DBservers" and "coordinators". The former actually store the data, the latter expose the database to the outside world. The clients talk to the coordinators exactly as they would talk to a single ArangoDB instance via the REST interface. The coordinators know about the configuration of the cluster and automatically forward the incoming requests to the right DBservers.

As a central highly available service to hold the cluster configuration and to synchronize reconfiguration and fail-over operations we currently use a an external program called etcd (see Github page). It provides a hierarchical key value store with strong consistency and reliability promises. This is called the "agency" and its processes are called "agents".

All this is admittedly a relatively complicated setup and involves a lot of steps for the startup and shutdown of clusters. Therefore we have created convenience functionality to plan, setup, start and shutdown clusters.

The whole process works in two phases, first the "planning" phase and then the "running" phase. In the planning phase it is decided which processes with which roles run on which machine, which ports they use, where the central agency resides and what ports its agents use. The result of the planning phase is a "cluster plan", which is just a relatively big data structure in JSON format. You can then use this cluster plan to startup, shutdown, check and cleanup your cluster.

This latter phase uses so-called "dispatchers". A dispatcher is yet another ArangoDB instance and you have to install exactly one such instance on every machine that will take part in your cluster. No special configuration whatsoever is needed and you can organize authentication exactly as you would in a normal ArangoDB instance. You only have to activate the dispatcher functionality in the configuration file (see options cluster.disable-dispatcher-kickstarter and cluster.disable-dispatcher-interface, which are both initially set to true in the standard setup we ship).

However, you can use any of these dispatchers to plan and start your cluster. In the planning phase, you have to tell the planner about all dispatchers in your cluster and it will automatically distribute your agency, DBserver and coordinator processes amongst the dispatchers. The result is the cluster plan which you feed into the kickstarter. The kickstarter is a program that actually uses the dispatchers to manipulate the processes in your cluster. It runs on one of the dispatchers, which analyses the cluster plan and executes those actions, for which it is personally responsible, and forwards all other actions to the corresponding dispatchers. This is possible, because the cluster plan incorporates the information about all dispatchers.

We also offer a graphical user interface to the cluster planner and dispatcher.

How to try it out

In this text we assume that you are working with a standard installation of ArangoDB with at least a version number of 2.0. This means that everything is compiled for cluster operation, that etcd is compiled and the executable is installed in the location mentioned in the configuration file. The first step is to switch on the dispatcher functionality in your configuration of arangod. In order to do this, change the cluster.disable-dispatcher-kickstarter and cluster.disable-dispatcher-interface options in arangod.conf both to false.

Note: Once you switch cluster.disable-dispatcher-interface to false, the usual web front end is automatically replaced with the web front end for cluster planning. Therefore you can simply point your browser to http://localhost:8529 (if you are running on the standard port) and you are guided through the planning and launching of a cluster with a graphical user interface. Alternatively, you can follow the instructions below to do the same on the command line interface.

We will first plan and launch a cluster, such that all your servers run on the local machine.

Start up a regular ArangoDB, either in console mode or connect to it with the Arango shell arangosh. Then you can ask it to plan a cluster for you:

arangodb> var Planner = require("org/arangodb/cluster").Planner;

arangodb> p = new Planner({numberOfDBservers:3, numberOfCoordinators:2});

[object Object]

If you are curious you can look at the plan of your cluster:

arangodb> p.getPlan();

This will show you a huge JSON document. More interestingly, some further components tell you more about the layout of your cluster:

arangodb> p.DBservers;

[

{

"id" : "Pavel",

"dispatcher" : "me",

"port" : 8629

},

{

"id" : "Perry",

"dispatcher" : "me",

"port" : 8630

},

{

"id" : "Pancho",

"dispatcher" : "me",

"port" : 8631

}

]

arangodb> p.coordinators;

[

{

"id" : "Claus",

"dispatcher" : "me",

"port" : 8530

},

{

"id" : "Chantalle",

"dispatcher" : "me",

"port" : 8531

}

]

This tells you the ports on which your ArangoDB processes will listen. We will need the 8530 (or whatever appears on your machine) for the coordinators below.

More interesting is that such a cluster plan document can be used to start up the cluster conveniently using a Kickstarter object. Please note that the launch method of the kickstarter shown below initializes all data directories and log files, so if you have previously used the same cluster plan you will lose all your data. Use the relaunch method described below instead in that case.

arangodb> var Kickstarter = require("org/arangodb/cluster").Kickstarter;

arangodb> k = new Kickstarter(p.getPlan());

arangodb> k.launch();

```js

That is all you have to do, to fire up your first cluster. You will see some

output, which you can safely ignore (as long as no error happens).

From that point on, you can contact one of the coordinators and use the cluster

as if it were a single ArangoDB instance (use the port number from above

instead of 8530, if you get a different one) (probably from another

shell window):

```js

$ arangosh --server.endpoint tcp://localhost:8530

[... some output omitted]

arangosh [_system]> db._listDatabases();

[

"_system"

]

```js

This for example, lists the cluster wide databases.

Now, let us create a sharded collection. Note, that we only have to specify

the number of shards to use in addition to the usual command.

The shards are automatically distributed among your DBservers:

```js

arangosh [_system]> example = db._create("example",{numberOfShards:6});

[ArangoCollection 1000001, "example" (type document, status loaded)]

arangosh [_system]> x = example.save({"name":"Hans", "age":44});

{

"error" : false,

"_id" : "example/1000008",

"_rev" : "13460426",

"_key" : "1000008"

}

arangosh [_system]> example.document(x._key);

{

"age" : 44,

"name" : "Hans",

"_id" : "example/1000008",

"_rev" : "13460426",

"_key" : "1000008"

}

```js

You can shut down your cluster by using the following Kickstarter

method (in the ArangoDB console):

```js

arangodb> k.shutdown();

If you want to start your cluster again without losing data you have previously stored in it, you can use the relaunch method in exactly the same way as you previously used the launch method:

arangodb> k.relaunch();

Note: If you have destroyed the object k for example because you have shutdown the ArangoDB instance in which you planned the cluster, then you can reproduce it for a relaunch operation, provided you have kept the cluster plan object provided by the getPlan method. If you had for example done:

arangodb> var plan = p.getPlan();

arangodb> require("fs").write("saved_plan.json",JSON.stringify(plan));

Then you can later do (in another session):

arangodb> var plan = require("fs").read("saved_plan.json");

arangodb> plan = JSON.parse(plan);

arangodb> var Kickstarter = require("org/arangodb/cluster").Kickstarter;

arangodb> var k = new Kickstarter(plan);

arangodb> k.relaunch();

to start the existing cluster anew.

You can check, whether or not, all your cluster processes are still running, by issuing:

arangodb> k.isHealthy();

This will show you the status of all processes in the cluster. You should see "RUNNING" there, in all the relevant places.

Finally, to clean up the whole cluster (losing all the data stored in it), do:

arangodb> k.shutdown();

arangodb> k.cleanup();

We conclude this section with another example using two machines, which will act as two dispatchers. We start from scratch using two machines, running on the network addresses tcp://192.168.173.78:8529 and tcp://192.168.173.13:6789. Both need to have a regular ArangoDB instance installed and running. Please make sure, that both bind to all network devices, so that they can talk to each other. Also enable the dispatcher functionality on both of them, as described above.

arangodb> var Planner = require("org/arangodb/cluster").Planner;

arangodb> var p = new Planner({

dispatchers: {"me":{"endpoint":"tcp://192.168.173.78:8529"},

"theother":{"endpoint":"tcp://192.168.173.13:6789"}},

"numberOfCoordinators":2, "numberOfDBservers": 2});

With these commands, you create a cluster plan involving two machines. The planner will put one DBserver and one Coordinator on each machine. You can now launch this cluster exactly as explained earlier:

arangodb> var Kickstarter = require("org/arangodb/cluster").Kickstarter;

arangodb> k = new Kickstarter(p.getPlan());

arangodb> k.launch();

Likewise, the methods shutdown, relaunch, isHealthy and cleanup work exactly as in the single server case.

See the corresponding chapter of the reference manual for detailed information about the Planner and Kickstarter classes.

Status of the implementation

This version 2.0 of ArangoDB contains the first usable implementation of the sharding extensions. However, not all planned features are included in this release. In particular, automatic fail-over is fully prepared in the architecture but is not yet implemented. If you use Version 2.0 in cluster mode in a production system, you have to organize failure recovery manually. This is why, at this stage with Version 2.0 we do not yet recommend to use the cluster mode in production systems. If you really need this feature now, please contact us.

This section provides an overview over the implemented and future features.

In normal single instance mode, ArangoDB works as usual with the same performance and functionality as in previous releases.

In cluster mode, the following things are implemented in version 2.0 and work:

- All basic CRUD operations for single documents and edges work essentially with good performance.

- One can use sharded collections and can configure the number of shards for each such collection individually. In particular, one can have fully sharded collections as well as cluster-wide available collections with only a single shard. After creation, these differences are transparent to the client.

- Creating and dropping cluster-wide databases works.

- Creating, dropping and modifying cluster-wide collections all work. Since these operations occur seldom, we will only improve their performance in a future release, when we will have our own implementation of the agency as well as a cluster-wide event managing system (see road map for release 2.3).

- Sharding in a collection, can be configured to use hashing on arbitrary properties of the documents in the collection.

- Creating and dropping indices on sharded collections works. Please note that an index on a sharded collection is not a global index but only leads to a local index of the same type on each shard.

- All SimpleQueries work. Again, we will improve the performance in future releases, when we revisit the AQL query optimizer (see road map for release 2.2).

- AQL queries work, but with relatively bad performance. Also, if the result of a query on a sharded collection is large, this can lead to an out of memory situation on the coordinator handling the request. We will improve this situation when we revisit the AQL query optimizer (see road map for release 2.2).

- Authentication on the cluster works with the method known from single ArangoDB instances on the coordinators. A new cluster-internal authorization scheme has been created. See below for hints on a sensible firewall and authorization setup.

- Most standard API calls of the REST interface work on the cluster as usual, with a few exceptions, which do no longer make sense on a cluster or are harder to implement. See below for details.

The following does not yet work, but is planned for future releases (see road map):

- Transactions can be run, but do not behave like transactions. They simply execute but have no atomicity or isolation in version 2.0. See the road map for version 2.X.

- Data-modification AQL queries are not executed atomically or isolated. If a data-modification AQL query fails for one shard, it might be rolled back there, but still complete on other shards.

- Data-modification AQL queries require a _key attribute in documents in order to operate. If a different shard key is chosen for a collection, specifying the _key attribute is currently still required. This restriction might be lifted in a future release.

- We plan to revise the AQL optimizer for version 2.2. This is necessary since for efficient queries in cluster mode we have to do as much as possible of the filtering and sorting on the individual DBservers rather than on the coordinator.

- Our software architecture is fully prepared for replication, automatic fail-over and recovery of a cluster, which will be implemented for version 2.3 (see our road map).

- This setup will at the same time, allow for hot swap and in-service maintenance and scaling of a cluster. However, in version 2.0 the cluster layout is static and no redistribution of data between the DBservers or moving of shards between servers is possible.

- At this stage the sharding of an edge collection is independent of the sharding of the corresponding vertex collection in a graph. For version 2.2 we plan to synchronize the two, to allow for more efficient graph traversal functions in large, sharded graphs. We will also do research on distributed algorithms for graphs and implement new algorithms in ArangoDB. However, at this stage, all graph traversal algorithms are executed on the coordinator and this means relatively poor performance since every single edge step leads to a network exchange.

- In version 2.0 the import API is broken for sharded collections. It will appear to work but will in fact silently fail. Fixing this is on the road map for version 2.1.

- In version 2.0 the arangodump and arangorestore programs can not be used talking to a coordinator to directly backup sharded collections. At this stage, one has to backup the DBservers individually using arangodump and arangorestore on them. The coordinators themselves do not hold any state and therefore do not need backup. Do not forget to backup the meta data stored in the agency because this is essential to access the sharded collections. These limitations will be fixed in version 2.1.

- In version 2.0 the replication API (/_api/replication) does not work on coordinators. This is intentional, since the plan is to organize replication with automatic fail-over directly on the DBservers, which is planned for version 2.3.

- The db..rotate() method for sharded collections is not yet implemented, but will be supported from version 2.1 onwards.

- The db..rename() method for sharded collections is not yet implemented, but will be supported from version 2.1 onwards.

- The db..checksum() method for sharded collections is not yet implemented, but will be supported from version 2.1 onwards.

The following restrictions will probably stay, for cluster mode, even in future versions. This is, because they are difficult or even impossible to implement efficiently:

- Custom key generators with the keyOptions property in the _create method for collections are not supported. We plan to improve this for version 2.1 (see road map). However, due to the distributed nature of a sharded collection, not everything that is possible in the single instance situation will be possible on a cluster. For example the auto-increment feature in a cluster with multiple DBservers and coordinators would have to lock the whole collection centrally for every document creation, which essentially defeats the performance purpose of sharding.

- Unique constraints on non-sharding keys are unsupported. The reason for this is that we do not plan to have global indices for sharded collections. Therefore, there is no single authority that could efficiently decide whether or not the unique constraint is satisfied by a new document. The only possibility would be to have a central locking mechanism and use heavy communication for every document creation to ensure the unique constraint.

- The method db..revision() for a sharded collection returns the highest revision number from all shards. However, revision numbers are assigned per shard, so this is not guaranteed to be the revision of the latest inserted document. Again, maintaining a global revision number over all shards is very difficult to maintain efficiently.

- The methods db..first() and db..last() are unsupported for collections with more than one shard. The reason for this, is that temporal order in a highly parallelized environment like a cluster is difficult or even impossible to achieve efficiently. In a cluster it is entirely possible that two different coordinators add two different documents to two different shards at the same time. In such a situation it is not even well-defined which of the two documents is "later". The only way to overcome this fundamental problem would again be a central locking mechanism, which is not desirable for performance reasons.

- Contrary to the situation in a single instance, objects representing sharded collections are broken after their database is dropped. In a future version they might report that they are broken, but it is not feasible and not desirable to retain the cluster database in the background until all collection objects are garbage collected.

- In a cluster, the automatic creation of collections on a call to db._save(ID) is not supported. This is because one would have no way to specify the number or distribution of shards for the newly created collection. Therefore we will not offer this feature for cluster mode.

Authentication in a cluster

In this section we describe, how authentication in a cluster is done properly. For experiments it is possible to run the cluster completely unauthorized by using the option --server.disable-authentication true on the command line or the corresponding entry in the configuration file. However, for production use, this is not desirable.

You can turn on authentication in the cluster by switching it on in the configuration of your dispatchers. When you now use the planner and kickstarter to create and launch a cluster, the arangod processes in your cluster will automatically run with authentication, exactly as the dispatchers themselves. However, the cluster will have a sharded collection _users with one shard containing only the user root with an empty password. We emphasize that this sharded cluster-wide collection is different from the _users collections in each dispatcher!

The coordinators in your cluster will use this cluster-wide sharded collection to authenticate HTTP requests. If you add users using the usual methods via a coordinator, you will in fact change the cluster-wide collection _users and thus all coordinators will eventually see the new users and authenticate against them. "Eventually" means that they might need a few seconds to notice the change in user setup and update their user cache.

The DBservers will have their authentication switched on as well. However, they do not use the cluster-wide _users collection for authentication, because the idea is, that the outside clients do not talk to the DBservers directly, but always go via the coordinators. For the cluster-internal communication between coordinators and DBservers (in both directions), we use a simpler setup: There are two new configuration options cluster.username and cluster.password, which default to root and the empty password "". If you want to deviate from this default you have to change these two configuration options in all configuration files on all machines in the cluster. This just means that you have to set these two options to the same values in all configuration files arangod.conf in all dispatchers, since the coordinators and DBservers will simply inherit this configuration file from the dispatcher that has launched them.

Let us summarize what you have to do, to enable authentication in a cluster:

1. Set server.disable-authentication to false in all configuration files of all dispatchers (this is already the default).

2. Put the same values for cluster.username and cluster.password in the very same configuration files of all dispatchers.

3. Create users via the usual interface on the coordinators (initially after the cluster launch there will be a single user root with empty password).

Please note, that in Version 2.0 of ArangoDB you can already configure the endpoints of the coordinators to use SSL. However, this is not yet conveniently supported in the planner, kickstarter and in the graphical cluster management tools. We will fix this in the next version.

Please also consider the comments in the following section about firewall setup.

Recommended firewall setup

This section is intended for people who run a cluster in production systems.

The whole idea of the cluster setup is that the coordinators serve HTTP requests to the outside world and that all other processes (DBservers and agency) are only available from within the cluster itself. Therefore, in a production environment, one has to put the whole cluster behind a firewall and only open the ports to the coordinators to the client processes.

Note however that for the asynchronous cluster-internal communication, the DBservers perform HTTP requests to the coordinators, which means that the coordinators must also be reachable from within the cluster.

Furthermore, it is of the utmost importance to hide the agent processes of the agency behind the firewall, since, at least at this stage, requests to them are completely unauthorized. Leaving their ports exposed to the outside world, endangers all data in the cluster, because everybody on the internet could make the cluster believe that, for example, you wanted your databases dropped! This weakness will be alleviated in future versions, because we will replace etcd by our own specialized agency implementation, which will allow for authentication.

A further comment applies to the dispatchers. Usually you will open the HTTP endpoints of your dispatchers to the outside world and switch on authentication for them. This is necessary to contact them from the outside, in the cluster launch phase. However, actually you only need to contact one of them, who will then in turn contact the others using cluster-internal communication. You can even get away with closing access to all dispatchers to the outside world, provided the machine running your browser is within the cluster network and does not have to go through the firewall to contact the dispatchers. It is important to be aware that anybody who can reach a dispatcher and can authorize himself to it can launch arbitrary processes on the machine on which the dispatcher runs!

Therefore we recommend to use SSL endpoints with user/password authentication on the dispatchers and to block access to them in the firewall. You then have to launch the cluster using an arangosh or browser running within the cluster.

Command-line options

Configuration Files

Options can be specified on the command line or in configuration files. If a string Variable occurs in the value, it is replaced by the corresponding environment variable.

General Options

General help

--help

-h

Prints a list of the most common options available and then exits. In order to see all options use --help-all.

Version

--version

-v

Prints the version of the server and exits.

Upgrade

--upgrade

Specifying this option will make the server perform a database upgrade at start. A database upgrade will first compare the version number stored in the file VERSION in the database directory with the current server version.

If the two version numbers match, the server will start normally.

If the version number found in the database directory is higher than the version number the server is running, the server expects this is an unintentional downgrade and will warn about this. It will however start normally. Using the server in these conditions is however not recommended nor supported.

If the version number found in the database directory is lower than the version number the server is running, the server will check whether there are any upgrade tasks to perform. It will then execute all required upgrade tasks and print their statuses. If one of the upgrade tasks fails, the server will exit and refuse to start. Re-starting the server with the upgrade option will then again trigger the upgrade check and execution until the problem is fixed. If all tasks are finished, the server will start normally.

Whether or not this option is specified, the server will always perform a version check on startup. Running the server with a non-matching version number in the VERSION file will make the server refuse to start.

Configuration

--configuration filename

-c filename

Specifies the name of the configuration file to use.

If this command is not passed to the server, then by default, the server will attempt to first locate a file named ~/.arango/arangod.conf in the user's home directory.

If no such file is found, the server will proceed to look for a file arangod.conf in the system configuration directory. The system configuration directory is platform-specific, and may be changed when compiling ArangoDB yourself. It may default to /etc/arangodb or /usr/local/etc/arangodb. This file is installed when using a package manager like rpm or dpkg. If you modify this file and later upgrade to a new version of ArangoDB, then the package manager normally warns you about the conflict. In order to avoid these warning for small adjustments, you can put local overrides into a file arangod.conf.local.

Only command line options with a value should be set within the configuration file. Command line options which act as flags should be entered on the command line when starting the server.

Whitespace in the configuration file is ignored. Each option is specified on a separate line in the form

key = value

Alternatively, a header section can be specified and options pertaining to that section can be specified in a shorter form

[log]

level = trace

rather than specifying

log.level = trace

Comments can be placed in the configuration file, only if the line begins with one or more hash symbols (#).

There may be occasions where a configuration file exists and the user wishes to override configuration settings stored in a configuration file. Any settings specified on the command line will overwrite the same setting when it appears in a configuration file. If the user wishes to completely ignore configuration files without necessarily deleting the file (or files), then add the command line option

-c none

or

--configuration none

When starting up the server. Note that, the word none is case-insensitive.

Daemon

--daemon

Runs the server as a daemon (as a background process). This parameter can only be set if the pid (process id) file is specified. That is, unless a value to the parameter pid-file is given, then the server will report an error and exit.

Default Language

--default-language default-language

The default language ist used for sorting and comparing strings. The language value is a two-letter language code (ISO-639) or it is composed by a two-letter language code with and a two letter country code (ISO-3166). Valid languages are "de", "en", "en_US" or "en_UK".

The default default-language is set to be the system locale on that platform.

Supervisor

--supervisor

Executes the server in supervisor mode. In the event that the server unexpectedly terminates due to an internal error, the supervisor will automatically restart the server. Setting this flag automatically implies that the server will run as a daemon. Note that, as with the daemon flag, this flag requires that the pid-file parameter will set.

unix> ./arangod --supervisor --pid-file /var/run/arangodb.pid /tmp/vocbase/

2012-06-27T15:58:28Z [10133] INFO starting up in supervisor mode

As can be seen (e.g. by executing the ps command), this will start a supervisor process and the actual database process:

unix> ps fax | grep arangod

10137 ?        Ssl    0:00 ./arangod --supervisor --pid-file /var/run/arangodb.pid /tmp/vocbase/

10142 ?        Sl     0:00  \_ ./arangod --supervisor --pid-file /var/run/arangodb.pid /tmp/vocbase/

When the database process terminates unexpectedly, the supervisor process will start up a new database process:

> kill -SIGSEGV 10142

> ps fax | grep arangod

10137 ?        Ssl    0:00 ./arangod --supervisor --pid-file /var/run/arangodb.pid /tmp/vocbase/

10168 ?        Sl     0:00  \_ ./arangod --supervisor --pid-file /var/run/arangodb.pid /tmp/vocbase/

User identity

--uid uid

The name (identity) of the user the server will run as. If this parameter is not specified, the server will not attempt to change its UID, so that the UID used by the server will be the same as the UID of the user who started the server. If this parameter is specified, then the server will change its UID after opening ports and reading configuration files, but before accepting connections or opening other files (such as recovery files). This is useful when the server must be started with raised privileges (in certain environments) but security considerations require that these privileges be dropped once the server has started work.

Observe that this parameter cannot be used to bypass operating system security. In general, this parameter (and its corresponding relative gid) can lower privileges but not raise them.

Group identity

--gid gid

The name (identity) of the group the server will run as. If this parameter is not specified, then the server will not attempt to change its GID, so that the GID the server runs as will be the primary group of the user who started the server. If this parameter is specified, then the server will change its GID after opening ports and reading configuration files, but before accepting connections or opening other files (such as recovery files).

This parameter is related to the parameter uid.

Process identity

--pid-file filename

The name of the process ID file to use when running the server as a daemon. This parameter must be specified if either the flag daemon or supervisor is set.

Console

--console

Runs the server in an exclusive emergency console mode. When starting the server with this option, the server is started with an interactive JavaScript emergency console, with all networking and HTTP interfaces of the server disabled.

No requests can be made to the server in this mode, and the only way to work with the server in this mode is by using the emergency console. Note that the server cannot be started in this mode if it is already running in this or another mode.

Random Generator

--random.generator arg

The argument is an integer (1,2,3 or 4) which sets the manner in which random numbers are generated. The default method (3) is to use the a non-blocking random (or pseudorandom) number generator supplied by the operating system.

Specifying an argument of 2, uses a blocking random (or pseudorandom) number generator. Specifying an argument 1 sets a pseudorandom number generator using an implication of the Mersenne Twister MT19937 algorithm. Algorithm 4 is a combination of the blocking random number generator and the Mersenne Twister.

Command-Line Options for arangod

Endpoint

--server.endpoint endpoint

Specifies an endpoint for HTTP requests by clients. Endpoints have the following pattern:

- tcp://ipv4-address:port - TCP/IP endpoint, using IPv4

- tcp://[ipv6-address]:port - TCP/IP endpoint, using IPv6

- ssl://ipv4-address:port - TCP/IP endpoint, using IPv4, SSL encryption

- ssl://[ipv6-address]:port - TCP/IP endpoint, using IPv6, SSL encryption

- unix:///path/to/socket - Unix domain socket endpoint

If a TCP/IP endpoint is specified without a port number, then the default port (8529) will be used. If multiple endpoints need to be used, the option can be repeated multiple times.

Examples

unix> ./arangod --server.endpoint tcp://127.0.0.1:8529

--server.endpoint ssl://127.0.0.1:8530

--server.keyfile server.pem /tmp/vocbase

2012-07-26T07:07:47Z [8161] INFO using SSL protocol version 'TLSv1'

2012-07-26T07:07:48Z [8161] INFO using endpoint 'ssl://127.0.0.1:8530' for http ssl requests

2012-07-26T07:07:48Z [8161] INFO using endpoint 'tcp://127.0.0.1:8529' for http tcp requests

2012-07-26T07:07:49Z [8161] INFO ArangoDB (version 1.1.alpha) is ready for business

2012-07-26T07:07:49Z [8161] INFO Have Fun!

Note: If you are using SSL-encrypted endpoints, you must also supply the path to a server certificate using the --server.keyfile option.

Endpoints can also be changed at runtime. Please refer to HTTP Interface for Endpoints for more details.

Reuse address

--server.reuse-address

If this boolean option is set to true then the socket option SO_REUSEADDR is set on all server endpoints, which is the default. If this option is set to false it is possible that it takes up to a minute after a server has terminated until it is possible for a new server to use the same endpoint again. This is why this is activated by default.

Please note however that under some operating systems this can be a security risk because it might be possible for another process to bind to the same address and port, possibly hijacking network traffic. Under Windows, ArangoDB additionally sets the flag SO_EXCLUSIVEADDRUSE as a measure to alleviate this problem.

Disable authentication

--server.disable-authentication

Setting value to true will turn off authentication on the server side so all clients can execute any action without authorization and privilege checks.

The default value is false.

Disable authentication-unix-sockets

--server.disable-authentication-unix-sockets value

Setting value to true will turn off authentication on the server side for requests coming in via UNIX domain sockets. With this flag enabled, clients located on the same host as the ArangoDB server can use UNIX domain sockets to connect to the server without authentication. Requests coming in by other means (e.g. TCP/IP) are not affected by this option.

The default value is false.

Note: this option is only available on platforms that support UNIX domain sockets.

Authenticate system only

--server.authenticate-system-only boolean

Controls whether incoming requests need authentication only if they are directed to the ArangoDB's internal APIs and features, located at /_api/, /_admin/ etc.

IF the flag is set to true, then HTTP authentication is only required for requests going to URLs starting with /_, but not for other URLs. The flag can thus be used to expose a user-made API without HTTP authentication to the outside world, but to prevent the outside world from using the ArangoDB API and the admin interface without authentication. Note that checking the URL is performed after any database name prefix has been removed. That means when the actual URL called is /_db/_system/myapp/myaction, the URL /myapp/myaction will be used for authenticate-system-only check.

The default is false.

Note that authentication still needs to be enabled for the server regularly in order for HTTP authentication to be forced for the ArangoDB API and the web interface. Setting only this flag is not enough.

You can control ArangoDB's general authentication feature with the --server.disable-authentication flag.

Disable replication-applier

--server.disable-replication-applier flag

If true the server will start with the replication applier turned off, even if the replication applier is configured with the autoStart option. Using the command-line option will not change the value of the autoStart option in the applier configuration, but will suppress auto-starting the replication applier just once.

If the option is not used, ArangoDB will read the applier configuration from the file REPLICATION-APPLIER-CONFIG on startup, and use the value of the autoStart attribute from this file.

The default is false.

Keep-alive timeout

--server.keep-alive-timeout

Allows to specify the timeout for HTTP keep-alive connections. The timeout value must be specified in seconds. Idle keep-alive connections will be closed by the server automatically when the timeout is reached. A keep-alive-timeout value 0 will disable the keep alive feature entirely.

Default API compatibility

--server.default-api-compatibility

This option can be used to determine the API compatibility of the ArangoDB server. It expects an ArangoDB version number as an integer, calculated as follows:

10000 \ major + 100 * minor (example: 10400 for ArangoDB 1.4)*

The value of this option will have an influence on some API return values when the HTTP client used does not send any compatibility information.

In most cases it will be sufficient to not set this option explicitly but to keep the default value. However, in case an "old" ArangoDB client is used that does not send any compatibility information and that cannot handle the responses of the current version of ArangoDB, it might be reasonable to set the option to an old version number to improve compatibility with older clients.

Hide Product header

--server.hide-product-header

If true, the server will exclude the HTTP header "Server: ArangoDB" in HTTP responses. If set to false, the server will send the header in responses.

The default is false.

Allow method override

--server.allow-method-override

When this option is set to true, the HTTP request method will optionally be fetched from one of the following HTTP request headers if present in the request:

- x-http-method

- x-http-method-override

- x-method-override

If the option is set to true and any of these headers is set, the request method will be overridden by the value of the header. For example, this allows issuing an HTTP DELETE request which to the outside world will look like an HTTP GET request. This allows bypassing proxies and tools that will only let certain request types pass.

Setting this option to true may impose a security risk so it should only be used in controlled environments.

The default value for this option is false.

Server threads

--server.threads number

Specifies the number of threads that are spawned to handle HTTP REST requests.

Keyfile

--server.keyfile filename

If SSL encryption is used, this option must be used to specify the filename of the server private key. The file must be PEM formatted and contain both the certificate and the server's private key.

The file specified by filename should have the following structure:

# create private key in file "server.key"

openssl genrsa -des3 -out server.key 1024

# create certificate signing request (csr) in file "server.csr"

openssl req -new -key server.key -out server.csr

# copy away original private key to "server.key.org"

cp server.key server.key.org

# remove passphrase from the private key

openssl rsa -in server.key.org -out server.key

# sign the csr with the key, creates certificate PEM file "server.crt"

openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt

# combine certificate and key into single PEM file "server.pem"

cat server.crt server.key > server.pem

You may use certificates issued by a Certificate Authority or self-signed certificates. Self-signed certificates can be created by a tool of your choice. When using OpenSSL for creating the self-signed certificate, the following commands should create a valid keyfile:

-----BEGIN CERTIFICATE-----

(base64 encoded certificate)

-----END CERTIFICATE-----

-----BEGIN RSA PRIVATE KEY-----

(base64 encoded private key)

-----END RSA PRIVATE KEY-----

For further information please check the manuals of the tools you use to create the certificate.

Note: the --server.keyfile option must be set if the server is started with at least one SSL endpoint.

Cafile

--server.cafile filename

This option can be used to specify a file with CA certificates that are sent to the client whenever the server requests a client certificate. If the file is specified, The server will only accept client requests with certificates issued by these CAs. Do not specify this option if you want clients to be able to connect without specific certificates.

The certificates in filename must be PEM formatted.

Note: this option is only relevant if at least one SSL endpoint is used.

SSL protocol

--server.ssl-protocolvalue

Use this option to specify the default encryption protocol to be used. The following variants are available:

- 1: SSLv2

- 2: SSLv23

- 3: SSLv3

- 4: TLSv1

- 5: TLSv1.2 (recommended)

The default value is 4 (i.e. TLSv1).

Note: this option is only relevant if at least one SSL endpoint is used.

SSL cache

--server.ssl-cache value

Set to true if SSL session caching should be used.

value has a default value of false (i.e. no caching).

Note: this option is only relevant if at least one SSL endpoint is used, and only if the client supports sending the session id.

SSL options

--server.ssl-options value

This option can be used to set various SSL-related options. Individual option values must be combined using bitwise OR.

Which options are available on your platform is determined by the OpenSSL version you use. The list of options available on your platform might be retrieved by the following shell command:

> grep "#define SSL_OP_.*" /usr/include/openssl/ssl.h

#define SSL_OP_MICROSOFT_SESS_ID_BUG                    0x00000001L

#define SSL_OP_NETSCAPE_CHALLENGE_BUG                   0x00000002L

#define SSL_OP_LEGACY_SERVER_CONNECT                    0x00000004L

#define SSL_OP_NETSCAPE_REUSE_CIPHER_CHANGE_BUG         0x00000008L

#define SSL_OP_SSLREF2_REUSE_CERT_TYPE_BUG              0x00000010L

#define SSL_OP_MICROSOFT_BIG_SSLV3_BUFFER               0x00000020L

...

A description of the options can be found online in the OpenSSL documentation

Note: this option is only relevant if at least one SSL endpoint is used.

SSL cipher

--server.ssl-cipher-list cipher-list

This option can be used to restrict the server to certain SSL ciphers only, and to define the relative usage preference of SSL ciphers.

The format of cipher-list is documented in the OpenSSL documentation.

To check which ciphers are available on your platform, you may use the following shell command:

> openssl ciphers -v

ECDHE-RSA-AES256-SHA    SSLv3 Kx=ECDH     Au=RSA  Enc=AES(256)  Mac=SHA1

ECDHE-ECDSA-AES256-SHA  SSLv3 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA1

DHE-RSA-AES256-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA1

DHE-DSS-AES256-SHA      SSLv3 Kx=DH       Au=DSS  Enc=AES(256)  Mac=SHA1

DHE-RSA-CAMELLIA256-SHA SSLv3 Kx=DH       Au=RSA  Enc=Camellia(256) Mac=SHA1

...

The default value for cipher-list is "ALL".

Note: this option is only relevant if at least one SSL endpoint is used.

Backlog size

--server.backlog-size

Allows to specify the size of the backlog for the listen system call The default value is 10. The maximum value is platform-dependent. Specifying a higher value than defined in the system header's SOMAXCONN may result in a warning on server start. The actual value used by listen may also be silently truncated on some platforms (this happens inside the listen system call).

Disable server statistics

--server.disable-statistics value

If this option is value is true, then ArangoDB's statistics gathering is turned off. Statistics gathering causes regular CPU activity so using this option to turn it off might relieve heavy-loaded instances. Note: this option is only available when ArangoDB has not been compiled with the option --disable-figures.

Session timeout

--server.session-timeout value

The timeout for web interface sessions, using for authenticating requests to the web interface (/_admin/aardvark) and related areas.

Sessions are only used when authentication is turned on.

Foxx queues

--server.foxx-queues flag

If true, the Foxx queues will be available and jobs in the queues will be executed asynchronously.

The default is true. When set to  false  the queue manager will be disabled and any jobs are prevented from being processed, which may reduce CPU load a great deal.

Foxx queues poll interval

--server.foxx-queues-poll-interval value

The poll interval for the Foxx queues manager. The value is specified in seconds. Lower values will mean more immediate and more frequent Foxx queue job execution, but will make the queue thread wake up and query the queues more often. When set to a low value, the queue thread might cause CPU load.

The default is 1 second. If Foxx queues are not used much, then this value may be increased to make the queues thread wake up less.

Directory

--database.directory directory

The directory containing the collections and datafiles. Defaults to /var/lib/arango. When specifying the database directory, please make sure the directory is actually writable by the arangod process.

You should further not use a database directory which is provided by a network filesystem such as NFS. The reason is that networked filesystems might cause inconsistencies when there are multiple parallel readers or writers or they lack features required by arangod (e.g. flock()).

directory

When using the command line version, you can simply supply the database directory as argument.

Examples

> ./arangod --server.endpoint tcp://127.0.0.1:8529 --database.directory /tmp/vocbase

Journal size

--database.maximal-journal-size size

Maximal size of journal in bytes. Can be overwritten when creating a new collection. Note that this also limits the maximal size of a single document.

The default is 32MB.

Wait for sync

--database.wait-for-sync boolean

Default wait-for-sync value. Can be overwritten when creating a new collection.

The default is false.

Force syncing of properties

--database.force-sync-properties boolean

Force syncing of collection properties to disk after creating a collection or updating its properties.

If turned off, no fsync will happen for the collection and database properties stored in  parameter.json  files in the file system. Turning off this option will speed up workloads that create and drop a lot of collections (e.g. test suites).

The default is true.

Disable AQL query tracking

--database.disable-query-tracking flag

If true, the server's query tracking feature will be disabled by default.

The default is false.

Throw collection not loaded error

--database.throw-collection-not-loaded-error flag

Accessing a not-yet loaded collection will automatically load a collection on first access. This flag controls what happens in case an operation would need to wait for another thread to finalize loading a collection. If set to true, then the first operation that accesses an unloaded collection will load it. Further threads that try to access the same collection while it is still loading will get an error (1238, collection not loaded). When the initial operation has completed loading the collection, all operations on the collection can be carried out normally, and error 1238 will not be thrown.

If set to false, the first thread that accesses a not-yet loaded collection will still load it. Other threads that try to access the collection while loading will not fail with error 1238 but instead block until the collection is fully loaded. This configuration might lead to all server threads being blocked because they are all waiting for the same collection to complete loading. Setting the option to true will prevent this from happening, but requires clients to catch error 1238 and react on it (maybe by scheduling a retry for later).

The default value is false.

AQL Query caching mode

--database.query-cache-mode

Toggles the AQL query cache behavior. Possible values are:

- off: do not use query cache

- on: always use query cache, except for queries that have their cache attribute set to false

- demand: use query cache only for queries that have their cache attribute set to true set

AQL Query cache size

--database.query-cache-max-results

Maximum number of query results that can be stored per database-specific query cache. If a query is eligible for caching and the number of items in the database's query cache is equal to this threshold value, another cached query result will be removed from the cache.

This option only has an effect if the query cache mode is set to either on or demand.

Index threads

--database.index-threads

Specifies the number of background threads for index creation. When a collection contains extra indexes other than the primary index, these other indexes can be built by multiple threads in parallel. The index threads are shared among multiple collections and databases. Specifying a value of 0 will turn off parallel building, meaning that indexes for each collection are built sequentially by the thread that opened the collection. If the number of index threads is greater than 1, it will also be used to built the edge index of a collection in parallel (this also requires the edge index in the collection to be split into multiple buckets).

V8 contexts

--server.v8-contexts number

Specifies the number of V8 contexts that are created for executing JavaScript code. More contexts allow execute more JavaScript actions in parallel, provided that there are also enough threads available. Please note that each V8 context will use a substantial amount of memory and requires periodic CPU processing time for garbage collection.

Garbage collection frequency (time-based)

--javascript.gc-frequency frequency

Specifies the frequency (in seconds) for the automatic garbage collection of JavaScript objects. This setting is useful to have the garbage collection still work in periods with no or little numbers of requests.

Garbage collection interval (request-based)

--javascript.gc-interval interval

Specifies the interval (approximately in number of requests) that the garbage collection for JavaScript objects will be run in each thread.

V8 options

--javascript.v8-options options

Optional arguments to pass to the V8 Javascript engine. The V8 engine will run with default settings unless explicit options are specified using this option. The options passed will be forwarded to the V8 engine which will parse them on its own. Passing invalid options may result in an error being printed on stderr and the option being ignored.

Options need to be passed in one string, with V8 option names being prefixed with double dashes. Multiple options need to be separated by whitespace. To get a list of all available V8 options, you can use the value "--help" as follows:

--javascript.v8-options "--help"

Another example of specific V8 options being set at startup:

--javascript.v8-options "--harmony --log"

Names and features or usable options depend on the version of V8 being used, and might change in the future if a different version of V8 is being used in ArangoDB. Not all options offered by V8 might be sensible to use in the context of ArangoDB. Use the specific options only if you are sure that they are not harmful for the regular database operation.

Write-ahead log options

Since ArangoDB 2.2, the server will write all data-modification operations into its write-ahead log.

The write-ahead log is a sequence of logfiles that are written in an append-only fashion. Full logfiles will eventually be garbage-collected, and the relevant data might be transferred into collection journals and datafiles. Unneeded and already garbage-collected logfiles will either be deleted or kept for the purpose of keeping a replication backlog.

Directory

--wal.directory

Specifies the directory in which the write-ahead logfiles should be stored. If this option is not specified, it defaults to the subdirectory journals in the server's global database directory. If the directory is not present, it will be created.

Logfile size

--wal.logfile-size

Specifies the filesize (in bytes) for each write-ahead logfile. The logfile size should be chosen so that each logfile can store a considerable amount of documents. The bigger the logfile size is chosen, the longer it will take to fill up a single logfile, which also influences the delay until the data in a logfile will be garbage-collected and written to collection journals and datafiles. It also affects how long logfile recovery will take at server start.

Allow oversize entries

--wal.allow-oversize-entries

Whether or not it is allowed to store individual documents that are bigger than would fit into a single logfile. Setting the option to false will make such operations fail with an error. Setting the option to true will make such operations succeed, but with a high potential performance impact. The reason is that for each oversize operation, an individual oversize logfile needs to be created which may also block other operations. The option should be set to false if it is certain that documents will always have a size smaller than a single logfile.

Suppress shape information

--wal.suppress-shape-information

Setting this variable to true will lead to no shape information being written into the write-ahead logfiles for documents or edges. While this is a good optimization for a single server to save memory (and disk space), it it will effectively disable using the write-ahead log as a reliable source for replicating changes to other servers. A master server with this option set to true will not be able to fully reproduce the structure of saved documents after a collection has been deleted. In case a replication client requests a document for which the collection is already deleted, the master will return an empty document. Note that this only affects replication and not normal operation on the master.

Do not set this variable to true on a server that you plan to use as a replication master

Number of reserve logfiles

--wal.reserve-logfiles

The maximum number of reserve logfiles that ArangoDB will create in a background process. Reserve logfiles are useful in the situation when an operation needs to be written to a logfile but the reserve space in the logfile is too low for storing the operation. In this case, a new logfile needs to be created to store the operation. Creating new logfiles is normally slow, so ArangoDB will try to pre-create logfiles in a background process so there are always reserve logfiles when the active logfile gets full. The number of reserve logfiles that ArangoDB keeps in the background is configurable with this option.

Number of historic logfiles

--wal.historic-logfiles

The maximum number of historic logfiles that ArangoDB will keep after they have been garbage-collected. If no replication is used, there is no need to keep historic logfiles except for having a local changelog.

In a replication setup, the number of historic logfiles affects the amount of data a slave can fetch from the master's logs. The more historic logfiles, the more historic data is available for a slave, which is useful if the connection between master and slave is unstable or slow. Not having enough historic logfiles available might lead to logfile data being deleted on the master already before a slave has fetched it.

Sync interval

--wal.sync-interval

The interval (in milliseconds) that ArangoDB will use to automatically synchronize data in its write-ahead logs to disk. Automatic syncs will only be performed for not-yet synchronized data, and only for operations that have been executed without the waitForSync attribute.

Throttling

--wal.throttle-when-pending

The maximum value for the number of write-ahead log garbage-collection queue elements. If set to 0, the queue size is unbounded, and no writtle-throttling will occur. If set to a non-zero value, writte-throttling will automatically kick in when the garbage-collection queue contains at least as many elements as specified by this option. While write-throttling is active, data-modification operations will intentionally be delayed by a configurable amount of time. This is to ensure the write-ahead log garbage collector can catch up with the operations executed. Write-throttling will stay active until the garbage-collection queue size goes down below the specified value. Write-throttling is turned off by default.

--wal.throttle-wait

This option determines the maximum wait time (in milliseconds) for operations that are write-throttled. If write-throttling is active and a new write operation is to be executed, it will wait for at most the specified amount of time for the write-ahead log garbage-collection queue size to fall below the throttling threshold. If the queue size decreases before the maximum wait time is over, the operation will be executed normally. If the queue size does not decrease before the wait time is over, the operation will be aborted with an error. This option only has an effect if  --wal.throttle-when-pending  has a non-zero value, which is not the default.

Number of slots

--wal.slots

Configures the amount of write slots the write-ahead log can give to write operations in parallel. Any write operation will lease a slot and return it to the write-ahead log when it is finished writing the data. A slot will remain blocked until the data in it was synchronized to disk. After that, a slot becomes reusable by following operations. The required number of slots is thus determined by the parallelity of write operations and the disk synchronization speed. Slow disks probably need higher values, and fast disks may only require a value lower than the default.

Ignore logfile errors

--wal.ignore-logfile-errors

Ignores any recovery errors caused by corrupted logfiles on startup. When set to false, the recovery procedure on startup will fail with an error whenever it encounters a corrupted (that includes only half-written) logfile. This is a security precaution to prevent data loss in case of disk errors etc. When the recovery procedure aborts because of corruption, any corrupted files can be inspected and fixed (or removed) manually and the server can be restarted afterwards.

Setting the option to true will make the server continue with the recovery procedure even in case it detects corrupt logfile entries. In this case it will stop at the first corrupted logfile entry and ignore all others, which might cause data loss.

Ignore recovery errors

--wal.ignore-recovery-errors

Ignores any recovery errors not caused by corrupted logfiles but by logical errors. Logical errors can occur if logfiles or any other server datafiles have been manually edited or the server is somehow misconfigured.

Ignore (non-WAL) datafile errors

--database.ignore-datafile-errors boolean

If set to  false , CRC mismatch and other errors in collection datafiles will lead to a collection not being loaded at all. The collection in this case becomes unavailable. If such collection needs to be loaded during WAL recovery, the WAL recovery will also abort (if not forced with option  --wal.ignore-recovery-errors true ).

Setting this flag to  false  protects users from unintentionally using a collection with corrupted datafiles, from which only a subset of the original data can be recovered. Working with such collection could lead to data loss and follow up errors. In order to access such collection, it is required to inspect and repair the collection datafile with the datafile debugger (arango-dfdb).

If set to  true , CRC mismatch and other errors during the loading of a collection will lead to the datafile being partially loaded, up to the position of the first error. All data up to until the invalid position will be loaded. This will enable users to continue with collection datafiles even if they are corrupted, but this will result in only a partial load of the original data and potential follow up errors. The WAL recovery will still abort when encountering a collection with a corrupted datafile, at least if  --wal.ignore-recovery-errors  is not set to  true .

The default value is false, so collections with corrupted datafiles will not be loaded at all, preventing partial loads and follow up errors. However, if such collection is required at server startup, during WAL recovery, the server will abort the recovery and refuse to start.

JavaScript Interface for managing Endpoints

The ArangoDB server can listen for incoming requests on multiple endpoints.

The endpoints are normally specified either in ArangoDB's configuration file or on the command-line, using the "--server.endpoint" option. The default endpoint for ArangoDB is tcp://127.0.0.1:8529 or tcp://localhost:8529. ArangoDB can also do a so called broadcast bind using tcp://0.0.0.0:8529. This way it will be reachable on all interfaces of the host. This may be useful on development systems that frequently change their network setup like laptops.

Each endpoint can optionally be restricted to a specific list of databases only, thus allowing the usage of different port numbers for different databases.

This may be useful in multi-tenant setups. A multi-endpoint setup may also be useful to turn on encrypted communication for just specific databases.

Endpoints equal TCP ports to be bound. On one specific ethernet interface each port can only be bound exactly once. You can look up your available interfaces using the ifconfig command on Linux / MacOSX - the Windows equivalent is ipconfig (See Wikipedia for more details). The general names of the interfaces differ on OS's and hardwares they run on. However, typically every host has a so called loopback interface, which is a virtual interface. By convention it always has the address 127.0.0.1 or ::1 (ipv6), and can only be reached from exactly the very same host. Ethernet interfaces usually have names like eth0, wlan0, eth1:17, le0 or a plain text name in Windows.

To find out which services already use ports (so ArangoDB can't bind them anymore), you can use the netstat command (it behaves a little different on each platform, run it with -lnpt on Linux, -p tcp on MacOSX or with -an on windows for valuable information).

The JavaScript interface for endpoints provides an operation for retrieving the list of endpoints. There is no functionality to add, remove or reconfigure an existing endpoint via the API after server start.

Please note that the operation to retrieve the list of endpoints can only be carried out in the default database (_system) and none of the other databases. When not in the default database, you must first switch to it using the db._useDatabase method.

List

db._listEndpoints()

Returns a list of all endpoints and their mapped databases.

Please note that managing endpoints can only be performed from out of the _system database. When not in the default database, you must first switch to it using the "db._useDatabase" method.

Command-Line Options for Clusters

Node ID

--cluster.my-local-info info

Some local information about the server in the cluster, this can for example be an IP address with a process ID or any string unique to the server. Specifying info is mandatory on startup if the server id (see below) is not specified. Each server of the cluster must have a unique local info. This is ignored if my-id below is specified.

Agency endpoint

--cluster.agency-endpoint endpoint

An agency endpoint the server can connect to. The option can be specified multiple times so the server can use a cluster of agency servers. Endpoints have the following pattern:

- tcp://ipv4-address:port - TCP/IP endpoint, using IPv4

- tcp://[ipv6-address]:port - TCP/IP endpoint, using IPv6

- ssl://ipv4-address:port - TCP/IP endpoint, using IPv4, SSL encryption

- ssl://[ipv6-address]:port - TCP/IP endpoint, using IPv6, SSL encryption

At least one endpoint must be specified or ArangoDB will refuse to start. It is recommended to specify at least two endpoints so ArangoDB has an alternative endpoint if one of them becomes unavailable.

Examples

--cluster.agency-endpoint tcp://192.168.1.1:4001 --cluster.agency-endpoint tcp://192.168.1.2:4002

Agency prefix

--cluster.agency-prefix prefix

The global key prefix used in all requests to the agency. The specified prefix will become part of each agency key. Specifying the key prefix allows managing multiple ArangoDB clusters with the same agency server(s).

prefix must consist of the letters a-z, A-Z and the digits 0-9 only. Specifying a prefix is mandatory.

Examples

--cluster.prefix mycluster

MyId

--cluster.my-id id

The local server's id in the cluster. Specifying id is mandatory on startup. Each server of the cluster must have a unique id.

Specifying the id is very important because the server id is used for determining the server's role and tasks in the cluster.

id must be a string consisting of the letters a-z, A-Z or the digits 0-9 only.

MyAddress

--cluster.my-address endpoint

The server's endpoint for cluster-internal communication. If specified, it must have the following pattern:

- tcp://ipv4-address:port - TCP/IP endpoint, using IPv4

- tcp://[ipv6-address]:port - TCP/IP endpoint, using IPv6

- ssl://ipv4-address:port - TCP/IP endpoint, using IPv4, SSL encryption

- ssl://[ipv6-address]:port - TCP/IP endpoint, using IPv6, SSL encryption

If no endpoint is specified, the server will look up its internal endpoint address in the agency. If no endpoint can be found in the agency for the server's id, ArangoDB will refuse to start.

Examples

--cluster.my-address tcp://192.168.1.1:8530

Username

--cluster.username username

The username used for authorization of cluster-internal requests. This username will be used to authenticate all requests and responses in cluster-internal communication, i.e. requests exchanged between coordinators and individual database servers.

This option is used for cluster-internal requests only. Regular requests to coordinators are authenticated normally using the data in the _users collection.

If coordinators and database servers are run with authentication turned off, (e.g. by setting the --server.disable-authentication option to true), the cluster-internal communication will also be unauthenticated.

Password

--cluster.password password

The password used for authorization of cluster-internal requests. This password will be used to authenticate all requests and responses in cluster-internal communication, i.e. requests exchanged between coordinators and individual database servers.

This option is used for cluster-internal requests only. Regular requests to coordinators are authenticated normally using the data in the  _users  collection.

If coordinators and database servers are run with authentication turned off, (e.g. by setting the --server.disable-authentication option to true), the cluster-internal communication will also be unauthenticated.

Command-Line Options for Logging

There are two different kinds of logs. Human-readable logs and machine-readable logs. The human-readable logs are used to provide an administration with information about the server. The machine-readable logs are used to provide statistics about executed requests and timings about computation steps.

General Logging Options

Logfile

--log.file filename

This option allows the user to specify the name of a file to which information is logged. By default, if no log file is specified, the standard output is used. Note that if the file named by filename does not exist, it will be created. If the file cannot be created (e.g. due to missing file privileges), the server will refuse to start. If the specified file already exists, output is appended to that file.

Use + to log to standard error. Use - to log to standard output. Use "" to disable logging to file.

--log.tty filename

Be default, if started on a tty, the log output will also go to the ttyp. Use "" to disable.

Request

--log.requests-file filename

This option allows the user to specify the name of a file to which requests are logged. By default, no log file is used and requests are not logged. Note that if the file named by filename does not exist, it will be created. If the file cannot be created (e.g. due to missing file privileges), the server will refuse to start. If the specified file already exists, output is appended to that file.

Use + to log to standard error. Use - to log to standard output. Use "" to disable request logging altogether.

The log format is

- "http-request" : static string indicating that an HTTP request was logged

- client address: IP address of client

- HTTP method type, e.g.  GET ,  POST

- HTTP version, e.g.  HTTP/1.1

- HTTP response code, e.g. 200

- request body length in bytes

- response body length in bytes

- server request processing time, containing the time span between fetching the first byte of the HTTP request and the start of the HTTP response

Human Readable Logging

Logfiles

Level

--log.level level

--log level

Allows the user to choose the level of information which is logged by the server. The argument level is specified as a string and can be one of the values listed below. Note that, fatal errors, that is, errors which cause the server to terminate, are always logged irrespective of the log level assigned by the user. The variant c log.level can be used in configuration files, the variant c log for command line options.

fatal: Logs errors which cause the server to terminate.

Fatal errors generally indicate some inconsistency with the manner in which the server has been coded. Fatal errors may also indicate a problem with the platform on which the server is running. Fatal errors always cause the server to terminate. For example,

2010-09-20T07:32:12Z [4742] FATAL a http server has already been created

error: Logs errors which the server has encountered.

These errors may not necessarily result in the termination of the server. For example,

2010-09-17T13:10:22Z [13967] ERROR strange log level 'errors'\, going to 'warning'

warning: Provides information on errors encountered by the server, which are not necessarily detrimental to it's continued operation.

For example,

2010-09-20T08:15:26Z [5533] WARNING got corrupted HTTP request 'POS?'

Note: The setting the log level to warning will also result in all errors to be logged as well.

info: Logs information about the status of the server.

For example,

2010-09-20T07:40:38Z [4998] INFO SimpleVOC ready for business

Note: The setting the log level to info will also result in all errors and warnings to be logged as well.

debug: Logs all errors, all warnings and debug information.

Debug log information is generally useful to find out the state of the server in the case of an error. For example,

2010-09-17T13:02:53Z [13783] DEBUG opened port 7000 for any

Note: The setting the log level to debug will also result in all errors, warnings and server status information to be logged as well.

trace: As the name suggests, logs information which may be useful to trace problems encountered with using the server.

For example,

2010-09-20T08:23:12Z [5687] TRACE trying to open port 8000

Note: The setting the log level to trace will also result in all errors, warnings, status information, and debug information to be logged as well.

Local Time

--log.use-local-time

If specified, all dates and times in log messages will use the server's local time-zone. If not specified, all dates and times in log messages will be printed in UTC / Zulu time. The date and time format used in logs is always  YYYY-MM-DD HH:MM:SS , regardless of this setting. If UTC time is used, a  Z  will be appended to indicate Zulu time.

Line number

--log.line-number

Normally, if an human readable fatal, error, warning or info message is logged, no information about the file and line number is provided. The file and line number is only logged for debug and trace message. This option can be use to always log these pieces of information.

Prefix

--log.prefix prefix

This option is used specify an prefix to logged text.

Thread

--log.thread

Whenever log output is generated, the process ID is written as part of the log information. Setting this option appends the thread id of the calling thread to the process id. For example,

2010-09-20T13:04:01Z [19355] INFO ready for business

when no thread is logged and

2010-09-20T13:04:17Z [19371-18446744072487317056] ready for business

when this command line option is set.

Source Filter

--log.source-filter arg

For debug and trace messages, only log those messages originated from the C source file arg. The argument can be used multiple times.

Content Filter

--log.content-filter arg

Only log message containing the specified string arg.

Performance

--log.performance

If this option is set, performance-related info messages will be logged via the regular logging mechanisms. These will consist of mostly timing and debugging information for performance-critical operations.

Currently performance-related operations are logged as INFO messages. Messages starting with prefix  [action]  indicate that an instrumented operation was started (note that its end won't be logged). Messages with prefix  [timer]  will contain timing information for operations. Note that no timing information will be logged for operations taking less time than 1 second. This is to ensure that sub-second operations do not pollute logs.

The contents of performance-related log messages enabled by this option are subject to change in future versions of ArangoDB.

Machine Readable Logging

Application

--log.application name

Specifies the name of the application which should be logged if this item of information is to be logged.

Facility

--log.facility name

If this option is set, then in addition to output being directed to the standard output (or to a specified file, in the case that the command line log.file option was set), log output is also sent to the system logging facility. The arg is the system log facility to use. See syslog for further details.

The value of arg depends on your syslog configuration. In general it will be user. Fatal messages are mapped to crit, so if arg is user, these messages will be logged as user.crit. Error messages are mapped to err. Warnings are mapped to warn. Info messages are mapped to notice. Debug messages are mapped to info. Trace messages are mapped to debug.

Command-Line Options for Communication

Scheduler threads

--scheduler.threads arg

An integer argument which sets the number of threads to use in the IO scheduler. The default is 1.

Scheduler maximal queue size

--scheduler.maximal-queue-size size

Specifies the maximum size of the dispatcher queue for asynchronous task execution. If the queue already contains size tasks, new tasks will be rejected until other tasks are popped from the queue. Setting this value may help preventing from running out of memory if the queue is filled up faster than the server can process requests.

Scheduler backend

--scheduler.backend arg

The I/O method used by the event handler. The default (if this option is not specified) is to try all recommended backends. This is platform specific. See libev for further details and the meaning of select, poll and epoll.

Io backends

--show-io-backends

If this option is specified, then the server will list available backends and exit. This option is useful only when used in conjunction with the option scheduler.backend. An integer is returned (which is platform dependent) which indicates available backends on your platform. See libev for further details and for the meaning of the integer returned. This describes the allowed integers for scheduler.backend, see here for details.

Authentication and Authorization

Authentication and Authorization

ArangoDB only provides a very simple authentication interface and no authorization. We plan to add authorization features in later releases, which will allow the administrator to restrict access to collections and queries to certain users, given them either read or write access.

Currently, you can only secure the access to ArangoDB in an all-or-nothing fashion. The collection _users contains all users and a salted SHA256 hash of their passwords. A user can be active or inactive. A typical document of this collection is

arangosh> db._users.toArray()

show execution results

Command-Line Options for the Authentication and Authorization

--server.disable-authentication

Setting value to true will turn off authentication on the server side so all clients can execute any action without authorization and privilege checks.

The default value is false.

--server.authenticate-system-only boolean

Controls whether incoming requests need authentication only if they are directed to the ArangoDB's internal APIs and features, located at /_api/, /_admin/ etc.

IF the flag is set to true, then HTTP authentication is only required for requests going to URLs starting with /_, but not for other URLs. The flag can thus be used to expose a user-made API without HTTP authentication to the outside world, but to prevent the outside world from using the ArangoDB API and the admin interface without authentication. Note that checking the URL is performed after any database name prefix has been removed. That means when the actual URL called is /_db/_system/myapp/myaction, the URL /myapp/myaction will be used for authenticate-system-only check.

The default is false.

Note that authentication still needs to be enabled for the server regularly in order for HTTP authentication to be forced for the ArangoDB API and the web interface. Setting only this flag is not enough.

You can control ArangoDB's general authentication feature with the --server.disable-authentication flag.

Introduction to User Management

ArangoDB provides basic functionality to add, modify and remove database users programmatically. The following functionality is provided by the users module and can be used from inside arangosh and arangod.

Note: This functionality is not available from within the web interface.

Save

users.save(user, passwd, active, extra, changePassword)

This will create a new ArangoDB user. The username must be specified in user and must not be empty.

The password must be given as a string, too, but can be left empty if required. If you pass the special value ARANGODB_DEFAULT_ROOT_PASSWORD, the password will be set the value stored in the environment variable  ARANGODB_DEFAULT_ROOT_PASSWORD . This can be used to pass an instance variable into ArangoDB. For example, the instance identifier from Amazon.

If the active attribute is not specified, it defaults to true. The extra attribute can be used to save custom data with the user.

If the changePassword attribute is not specified, it defaults to false. The changePassword attribute can be used to indicate that the user must change has password before logging in.

This method will fail if either the username or the passwords are not specified or given in a wrong format, or there already exists a user with the specified name.

The new user account can only be used after the server is either restarted or the server authentication cache is reloaded.

Note: this function will not work from within the web interface

Examples

arangosh> require("org/arangodb/users").save("my-user", "my-secret-password");

show execution results

Reload

users.reload()

Reloads the user authentication data on the server

All user authentication data is loaded by the server once on startup only and is cached after that. When users get added or deleted, a cache flush is required, and this can be performed by called this method.

Examples

arangosh> require("org/arangodb/users").reload();

Note: this function will not work from within the web interface

Document

users.document(user)

Fetches an existing ArangoDB user from the database.

The username must be specified in user.

This method will fail if the user cannot be found in the database.

Examples

arangosh> require("org/arangodb/users").document("my-user");

show execution results

Note: this function will not work from within the web interface

Replace

users.replace(user, passwd, active, extra, changePassword)

This will look up an existing ArangoDB user and replace its user data.

The username must be specified in user, and a user with the specified name must already exist in the database.

The password must be given as a string, too, but can be left empty if required.

If the active attribute is not specified, it defaults to true. The extra attribute can be used to save custom data with the user.

If the changePassword attribute is not specified, it defaults to false. The changePassword attribute can be used to indicate that the user must change has password before logging in.

This method will fail if either the username or the passwords are not specified or given in a wrong format, or if the specified user cannot be found in the database.

Note: this function will not work from within the web interface

Examples

arangosh> require("org/arangodb/users").replace("my-user", "my-changed-password");

show execution results

Update

users.update(user, passwd, active, extra, changePassword)

This will update an existing ArangoDB user with a new password and other data.

The username must be specified in user and the user must already exist in the database.

The password must be given as a string, too, but can be left empty if required.

If the active attribute is not specified, the current value saved for the user will not be changed. The same is true for the extra and the changePassword attribute.

This method will fail if either the username or the passwords are not specified or given in a wrong format, or if the specified user cannot be found in the database.

Note: this function will not work from within the web interface

Examples

arangosh> require("org/arangodb/users").update("my-user", "my-secret-password");

show execution results

isValid

users.isValid(user, password)

Checks whether the given combination of username and password is valid. The function will return a boolean value if the combination of username and password is valid.

Each call to this function is penalized by the server sleeping a random amount of time.

Examples

arangosh> require("org/arangodb/users").isValid("my-user", "my-secret-password");

true

Note: this function will not work from within the web interface

all()

users.all()

Fetches all existing ArangoDB users from the database.

Examples

arangosh> require("org/arangodb/users").all();

show execution results

Remove

users.remove(user)

Removes an existing ArangoDB user from the database.

The username must be specified in User and the specified user must exist in the database.

This method will fail if the user cannot be found in the database.

Note: this function will not work from within the web interface

Examples

arangosh> require("org/arangodb/users").remove("my-user");

Emergency Console

In Case Of Disaster

The following command starts an emergency console.

Note: Never start the emergency console for a database which also has a server attached to it. In general the ArangoDB shell is what you want.

> ./arangod --console --log error /tmp/vocbase

ArangoDB shell [V8 version 3.9.4, DB version 1.x.y]

arango> 1 + 2;

3

arango> var db = require("org/arangodb").db; db.geo.count();

703

The emergency console provides a JavaScript console directly running in the arangod server process. This allows to debug and examine collections and documents as with the normal ArangoDB shell, but without client/server communication.

However, it is very likely that you will never need the emergency console unless you are an ArangoDB developer.

HTTP Interface

Following you have ArangoDB's HTTP Interface for Documents, Databases, Edges and more.

There are also some examples provided for every API action.

You may also use the interactive SWAGGER documentation in the ArangoDB webinterface in Links -> API Documentation to explore the API calls below.

General HTTP Request Handling in ArangoDB

Protocol

ArangoDB exposes its API via HTTP, making the server accessible easily with a variety of clients and tools (e.g. browsers, curl, telnet). The communication can optionally be SSL-encrypted.

ArangoDB uses the standard HTTP methods (e.g. GET, POST, PUT, DELETE) plus the PATCH method described in RFC 5789.

Most server APIs expect clients to send any payload data in JSON format. Details on the expected format and JSON attributes can be found in the documentation of the individual server methods.

Clients sending requests to ArangoDB must use either HTTP 1.0 or HTTP 1.1. Other HTTP versions are not supported by ArangoDB and any attempt to send a different HTTP version signature will result in the server responding with an HTTP 505 (HTTP version not supported) error.

ArangoDB will always respond to client requests with HTTP 1.1. Clients should therefore support HTTP version 1.1.

Clients are required to include the Content-Length HTTP header with the correct content length in every request that can have a body (e.g. POST, PUT or PATCH) request. ArangoDB will not process requests without a Content-Length header - thus chunked transfer encoding for POST-documents is not supported.

HTTP Keep-Alive

ArangoDB supports HTTP keep-alive. If the client does not send a Connection header in its request, and the client uses HTTP version 1.1, ArangoDB will assume the client wants to keep alive the connection. If clients do not wish to use the keep-alive feature, they should explicitly indicate that by sending a Connection: Close HTTP header in the request.

ArangoDB will close connections automatically for clients that send requests using HTTP 1.0, except if they send an Connection: Keep-Alive header.

The default Keep-Alive timeout can be specified at server start using the --server.keep-alive-timeout parameter.

Establishing TCP connections is expensive, since it takes several ping pongs between the communication parties. Therefore you can use connection keepalive to send several HTTP request over one TCP-connection; Each request is treated independently by definition. You can use this feature to build up a so called connection pool with several established connections in your client application, and dynamically re-use one of those then idle connections for subsequent requests.

Blocking vs. Non-blocking HTTP Requests

ArangoDB supports both blocking and non-blocking HTTP requests.

ArangoDB is a multi-threaded server, allowing the processing of multiple client requests at the same time. Request/response handling and the actual work are performed on the server in parallel by multiple worker threads.

Still, clients need to wait for their requests to be processed by the server, and thus keep one connection of a pool occupied. By default, the server will fully process an incoming request and then return the result to the client when the operation is finished. The client must wait for the server's HTTP response before it can send additional requests over the same connection. For clients that are single-threaded and/or are blocking on I/O themselves, waiting idle for the server response may be non-optimal.

To reduce blocking on the client side, ArangoDB offers a generic mechanism for non-blocking, asynchronous execution: clients can add the HTTP header x-arango-async: true to any of their requests, marking them as to be executed asynchronously on the server. ArangoDB will put such requests into an in-memory task queue and return an HTTP 202 (accepted) response to the client instantly and thus finish this HTTP-request. The server will execute the tasks from the queue asynchronously as fast as possible, while clients can continue to do other work. If the server queue is full (i.e. contains as many tasks as specified by the option "--scheduler.maximal-queue-size", then the request will be rejected instantly with an HTTP 500 (internal server error) response.

Asynchronous execution decouples the request/response handling from the actual work to be performed, allowing fast server responses and greatly reducing wait time for clients. Overall this allows for much higher throughput than if clients would always wait for the server's response.

Keep in mind that the asynchronous execution is just "fire and forget". Clients will get any of their asynchronous requests answered with a generic HTTP 202 response. At the time the server sends this response, it does not know whether the requested operation can be carried out successfully (the actual operation execution will happen at some later point). Clients therefore cannot make a decision based on the server response and must rely on their requests being valid and processable by the server.

Additionally, the server's asynchronous task queue is an in-memory data structure, meaning not-yet processed tasks from the queue might be lost in case of a crash. Clients should therefore not use the asynchronous feature when they have strict durability requirements or if they rely on the immediate result of the request they send.

Authentication

Client authentication can be achieved by using the Authorization HTTP header in client requests. ArangoDB supports HTTP Basic authentication.

Authentication is optional. To enforce authentication for incoming requested, the server must be started with the option --server.disable-authentication. Please note that requests using the HTTP OPTIONS method will be answered by ArangoDB in any case, even if no authentication data is sent by the client or if the authentication data is wrong. This is required for handling CORS preflight requests (see Cross Origin Resource Sharing requests). The response to an HTTP OPTIONS request will be generic and not expose any private data.

Please note that when authentication is turned on in ArangoDB, it will by default affect all incoming requests.

There is an additional option -server.authenticate-system-only "--server.authenticate-system-only" to restrict authentication to requests to the ArangoDB internal APIs and the admin interface. This option can be used to expose a public API built with ArangoDB to the outside world without the need for HTTP authentication, but to still protect the usage of the ArangoDB API (i.e. /_api/**) and the admin interface (i.e. /_admin/**) with HTTP authentication.

If the server is started with the --server.authenticate-system-only parameter set to false (which is the default), all incoming requests need HTTP authentication if the server is configured to require HTTP authentication. Setting the option to false will make the server require authentication only for requests to the internal functionality at /_api/ or /_admin and will allow unauthenticated requests to all other URLs.

Whenever authentication is required and the client has not yet authenticated, ArangoDB will return HTTP 401 (Unauthorized). It will also send the WWW-Authenticate response header, indicating that the client should prompt the user for username and password if supported. If the client is a browser, then sending back this header will normally trigger the display of the browser-side HTTP authentication dialog. As showing the browser HTTP authentication dialog is undesired in AJAX requests, ArangoDB can be told to not send the WWW-Authenticate header back to the client. Whenever a client sends the X-Omit-WWW-Authenticate HTTP header (with an arbitrary value) to ArangoDB, ArangoDB will only send status code 401, but no WWW-Authenticate header. This allows clients to implement credentials handling and bypassing the browser's built-in dialog.

Error Handling

The following should be noted about how ArangoDB handles client errors in its HTTP layer:

- client requests using an HTTP version signature different than HTTP/1.0 or HTTP/1.1 will get an HTTP 505 (HTTP version not supported) error in return.

- ArangoDB will reject client requests with a negative value in the Content-Length request header with HTTP 411 (Length Required).

- Arangodb doesn't support POST with transfer-encoding: chunked which forbids the Content-Length header above.

- the maximum URL length accepted by ArangoDB is 16K. Incoming requests with longer URLs will be rejected with an HTTP 414 (Request-URI too long) error.

- if the client sends a Content-Length header with a value bigger than 0 for an HTTP GET, HEAD, or DELETE request, ArangoDB will process the request, but will write a warning to its log file.

- when the client sends a Content-Length header that has a value that is lower than the actual size of the body sent, ArangoDB will respond with HTTP 400 (Bad Request).

- if clients send a Content-Length value bigger than the actual size of the body of the request, ArangoDB will wait for about 90 seconds for the client to complete its request. If the client does not send the remaining body data within this time, ArangoDB will close the connection. Clients should avoid sending such malformed requests as this will block one tcp connection, and may lead to a temporary filedescriptor leak.

- when clients send a body or a Content-Length value bigger than the maximum allowed value (512 MB), ArangoDB will respond with HTTP 413 (Request Entity Too Large).

- if the overall length of the HTTP headers a client sends for one request exceeds the maximum allowed size (1 MB), the server will fail with HTTP 431 (Request Header Fields Too Large).

- if clients request an HTTP method that is not supported by the server, ArangoDB will return with HTTP 405 (Method Not Allowed). ArangoDB offers general support for the following HTTP methods:

- GET

- POST

- PUT

- DELETE

- HEAD

- PATCH

- OPTIONS

Please note that not all server actions allow using all of these HTTP methods. You should look up up the supported methods for each method you intend to use in the manual.

Requests using any other HTTP method (such as for example CONNECT, TRACE etc.) will be rejected by ArangoDB as mentioned before.

Cross Origin Resource Sharing (CORS) requests

ArangoDB will automatically handle CORS requests as follows:

- when the client sends an Origin HTTP header, ArangoDB will return a header access-control-allow-origin containing the value the client sent in the Origin header.

- for non-trivial CORS requests, clients may issue a preflight request via an additional HTTP OPTIONS request. ArangoDB will automatically answer such preflight HTTP OPTIONS requests with an HTTP 200 response with an empty body. ArangoDB will return the following headers in the response:

- access-control-allow-origin: will contain the value that the client provided in the Origin header of the request

- access-control-allow-methods: will contain an array of all HTTP methods generally supported by ArangoDB. This array does not depend on the URL the client requested and is the same for all CORS requests.

- access-control-allow-headers: will contain exactly the value that the client has provided in the Access-Control-Request-Header header of the request. This header will only be returned if the client has specified the header in the request. ArangoDB will send back the original value without further validation.

- access-control-max-age: will return a cache lifetime for the preflight response as determined by ArangoDB.

- any access-control-allow-credentials header sent by the client is ignored by ArangoDB its value is not true. If a client sends a header value of true, ArangoDB will return the header access-control-allow-credentials: true, too.

Note that CORS preflight requests will probably not send any authentication data with them. One of the purposes of the preflight request is to check whether the server accepts authentication or not.

A consequence of this is that ArangoDB will allow requests using the HTTP OPTIONS method without credentials, even when the server is run with authentication enabled.

The response to the HTTP OPTIONS request will however be a generic response that will not expose any private data and thus can be considered "safe" even without credentials.

HTTP method overriding

ArangoDB provides a startup option --server.allow-method-override. This option can be set to allow overriding the HTTP request method (e.g. GET, POST, PUT, DELETE, PATCH) of a request using one of the following custom HTTP headers:

- x-http-method-override

- x-http-method

- x-method-override

This allows using HTTP clients that do not support all "common" HTTP methods such as PUT, PATCH and DELETE. It also allows bypassing proxies and tools that would otherwise just let certain types of requests (e.g. GET and POST) pass through.

Enabling this option may impose a security risk, so it should only be used in very controlled environments. Thus the default value for this option is false (no method overriding allowed). You need to enable it explicitly if you want to use this feature.

HTTP Interface for Databases

Address of a Database

Any operation triggered via ArangoDB's HTTP REST API is executed in the context of exactly one database. To explicitly specify the database in a request, the request URI must contain the database name in front of the actual path:

http://localhost:8529/_db/mydb/...

where ... is the actual path to the accessed resource. In the example, the resource will be accessed in the context of the database mydb. Actual URLs in the context of mydb could look like this:

http://localhost:8529/_db/mydb/_api/version

http://localhost:8529/_db/mydb/_api/document/test/12345

http://localhost:8529/_db/mydb/myapp/get

Database-to-Endpoint Mapping

If a database name is present in the URI as above, ArangoDB will consult the database-to-endpoint mapping for the current endpoint, and validate if access to the database is allowed on the endpoint. If the endpoint is not restricted to an array of databases, ArangoDB will continue with the regular authentication procedure. If the endpoint is restricted to an array of specified databases, ArangoDB will check if the requested database is in the array. If not, the request will be turned down instantly. If yes, then ArangoDB will continue with the regular authentication procedure.

If the request URI was http:// localhost:8529/_db/mydb/..., then the request to mydb will be allowed (or disallowed) in the following situations:

Endpoint-to-database mapping           Access to *mydb* allowed?

----------------------------           -------------------------

[ ]                                    yes

[ "_system" ]                          no

[ "_system", "mydb" ]                  yes

[ "mydb" ]                             yes

[ "mydb", "_system" ]                  yes

[ "test1", "test2" ]                   no

In case no database name is specified in the request URI, ArangoDB will derive the database name from the endpoint-to-database mapping of the endpoint the connection was coming in on.

If the endpoint is not restricted to an array of databases, ArangoDB will assume the _system database. If the endpoint is restricted to one or multiple databases, ArangoDB will assume the first name from the array.

Following is an overview of which database name will be assumed for different endpoint-to-database mappings in case no database name is specified in the URI:

Endpoint-to-database mapping           Database

----------------------------           --------

[ ]                                    _system

[ "_system" ]                          _system

[ "_system", "mydb" ]                  _system

[ "mydb" ]                             mydb

[ "mydb", "_system" ]                  mydb

Database Management

This is an introduction to ArangoDB's HTTP interface for managing databases.

The HTTP interface for databases provides operations to create and drop individual databases. These are mapped to the standard HTTP methods POST and DELETE. There is also the GET method to retrieve an array of existing databases.

Please note that all database management operations can only be accessed via the default database (_system) and none of the other databases.

Managing Databases using HTTP

Information of the database

retrieves information about the current database

GET /_api/database/current

Retrieves information about the current database

The response is a JSON object with the following attributes:

- name: the name of the current database

- id: the id of the current database

- path: the filesystem path of the current database

- isSystem: whether or not the current database is the _system database

Return Codes

- 200: is returned if the information was retrieved successfully.

- 400: is returned if the request is invalid.

- 404: is returned if the database could not be found.

Examples

shell> curl --dump - http://localhost:8529/_api/database/current

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

List of accessible databases

retrieves a list of all databases the current user can access

GET /_api/database/user

Retrieves the list of all databases the current user can access without specifying a different username or password.

Return Codes

- 200: is returned if the list of database was compiled successfully.

- 400: is returned if the request is invalid.

Examples

shell> curl --dump - http://localhost:8529/_api/database/user

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

List of databases

retrieves a list of all existing databases

GET /_api/database

Retrieves the list of all existing databases

Note: retrieving the list of databases is only possible from within the _system database.

Return Codes

- 200: is returned if the list of database was compiled successfully.

- 400: is returned if the request is invalid.

- 403: is returned if the request was not executed in the _system database.

Examples

shell> curl --dump - http://localhost:8529/_api/database

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Create database

creates a new database

POST /_api/database

Body Parameters

- username: The user name as a string. If users is not specified or does not contain any users, a default user root will be created with an empty string password. This ensures that the new database will be accessible after it is created.

- name: Has to contain a valid database name.

- extra: A JSON object with extra user information. The data contained in extra will be stored for the user but not be interpreted further by ArangoDB.

- passwd: The user password as a string. If not specified, it will default to an empty string.

- active: A Flag indicating whether the user account should be activated or not. The default value is true.

- users: Has to be a list of user objects to initially create for the new database. Each user object can contain the following attributes:

- active: if False the user won't be able to log into the database.

- passwd: Password for the user

- username: Loginname of the user to be created

Creates a new database

The response is a JSON object with the attribute result set to true.

Note: creating a new database is only possible from within the _system database.

Return Codes

- 201: is returned if the database was created successfully.

- 400: is returned if the request parameters are invalid or if a database with the specified name already exists.

- 403: is returned if the request was not executed in the _system database.

- 409: is returned if a database with the specified name already exists.

Examples

Creating a database named example.

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/database <<EOF

{

"name" : "example"

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Creating a database named mydb with two users.

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/database <<EOF

{

"name" : "mydb",

"users" : [

{

"username" : "admin",

"passwd" : "secret",

"active" : true

},

{

"username" : "tester",

"passwd" : "test001",

"active" : false

}

]

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Drop database

drop an existing database

DELETE /_api/database/{database-name}

Path Parameters

- database-name (required): The name of the database

Drops the database along with all data stored in it.

Note: dropping a database is only possible from within the _system database. The _system database itself cannot be dropped.

Return Codes

- 200: is returned if the database was dropped successfully.

- 400: is returned if the request is malformed.

- 403: is returned if the request was not executed in the _system database.

- 404: is returned if the database could not be found.

Examples

shell> curl -X DELETE --dump - http://localhost:8529/_api/database/example

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Notes on Databases

Please keep in mind that each database contains its own system collections, which need to set up when a database is created. This will make the creation of a database take a while. Replication is configured on a per-database level, meaning that any replication logging or applying for the a new database must be configured explicitly after a new database has been created. Foxx applications are also available only in the context of the database they have been installed in. A new database will only provide access to the system applications shipped with ArangoDB (that is the web interface at the moment) and no other Foxx applications until they are explicitly installed for the particular database.

Database

ArangoDB can handle multiple databases in the same server instance. Databases can be used to logically group and separate data. An ArangoDB database consists of collections and dedicated database-specific worker processes. A database contains its own collections (which cannot be accessed from other databases), Foxx applications and replication loggers and appliers. Each ArangoDB database contains its own system collections (e.g. _users, _graphs, ...).

There will always be at least one database in ArangoDB. This is the default database named _system. This database cannot be dropped and provides special operations for creating, dropping and enumerating databases. Users can create additional databases and give them unique names to access them later. Database management operations cannot be initiated from out of user-defined databases.

When ArangoDB is accessed via its HTTP REST API, the database name is read from the first part of the request URI path (e.g. /_db/_system/...). If the request URI does not contain a database name, the database name is automatically determined by the algorithm described in Database-to-Endpoint Mapping .

Database Name

A single ArangoDB instance can handle multiple databases in parallel. When multiple databases are used, each database must be given an unique name. This name is used to uniquely identify a database. The default database in ArangoDB is named system. The database name is a string consisting of only letters, digits and the (underscore) and - (dash) characters. User-defined database names must always start with a letter. Database names are case-sensitive.

Database Organization

A single ArangoDB instance can handle multiple databases in parallel. By default, there will be at least one database which is named _system. Databases are physically stored in separate sub-directories underneath the database directory, which itself resides in the instance's data directory.

Each database has its own sub-directory, named database-. The database directory contains sub-directories for the collections of the database, and a file named parameter.json. This file contains the database id and name.

In an example ArangoDB instance which has two databases, the filesystem layout could look like this:

data/                     # the instance's data directory

databases/              # sub-directory containing all databases' data

database-<id>/        # sub-directory for a single database

parameter.json      # file containing database id and name

collection-<id>/    # directory containing data about a collection

database-<id>/        # sub-directory for another database

parameter.json      # file containing database id and name

collection-<id>/    # directory containing data about a collection

collection-<id>/    # directory containing data about a collection

Foxx applications are also organized in database-specific directories inside the application path. The filesystem layout could look like this:

apps/                   # the instance's application directory

system/               # system applications (can be ignored)

databases/            # sub-directory containing database-specific applications

<database-name>/    # sub-directory for a single database

<app-name>        # sub-directory for a single application

<app-name>        # sub-directory for a single application

<database-name>/    # sub-directory for another database

<app-name>        # sub-directory for a single application

`

HTTP Interface for Documents

Documents, Identifiers, Handles

This is an introduction to ArangoDB's REST interface for documents.

Documents in ArangoDB are JSON objects. These objects can be nested (to any depth) and may contain lists. Each document is uniquely identified by its document handle.

An example document:

{

"_id" : "myusers/3456789",

"_key" : "3456789",

"_rev" : "3456789",

"firstName" : "Hugo",

"lastName" : "Schlonz",

"address" : {

"street" : "Street of Happiness",

"city" : "Heretown"

},

"hobbies" : [

"swimming",

"biking",

"programming"

]

}

All documents contain special attributes: the document handle in  _id , the document's unique key in  _key  and and the Etag aka document revision in  _rev . The value of the  _key  attribute can be specified by the user when creating a document.  _id  and  _key  values are immutable once the document has been created. The  _rev  value is maintained by ArangoDB autonomously.

Document Handle

A document handle uniquely identifies a document in the database. It is a string and consists of the collection's name and the document key ( _key  attribute) separated by  / .

Document Key

A document key uniquely identifies a document in the collection it it stored in. It can and should be used by clients when specific documents are searched. Document keys are stored in the  _key  attribute of documents. The key values are automatically indexed by ArangoDB in a collection's primary index. Thus looking up a document by its key is regularly a fast operation. The _key value of a document is immutable once the document has been created. By default, ArangoDB will auto-generate a document key if no _key attribute is specified, and use the user-specified _key otherwise.

This behavior can be changed on a per-collection level by creating collections with the keyOptions attribute.

Using keyOptions it is possible to disallow user-specified keys completely, or to force a specific regime for auto-generating the _key values.

Document Revision

As ArangoDB supports MVCC, documents can exist in more than one revision. The document revision is the MVCC token used to identify a particular revision of a document. It is a string value currently containing an integer number and is unique within the list of document revisions for a single document. Document revisions can be used to conditionally update, replace or delete documents in the database. In order to find a particular revision of a document, you need the document handle and the document revision. ArangoDB currently uses 64bit unsigned integer values to maintain document revisions internally. When returning document revisions to clients, ArangoDB will put them into a string to ensure the revision id is not clipped by clients that do not support big integers. Clients should treat the revision id returned by ArangoDB as an opaque string when they store or use it locally. This will allow ArangoDB to change the format of revision ids later if this should be required. Clients can use revisions ids to perform simple equality/non-equality comparisons (e.g. to check whether a document has changed or not), but they should not use revision ids to perform greater/less than comparisons with them to check if a document revision is older than one another, even if this might work for some cases.

Note: Revision ids have been returned as integers up to including ArangoDB 1.1

Document Etag

The document revision enclosed in double quotes. The revision is returned by several HTTP API methods in the Etag HTTP header.

The basic operations (create, read, update, delete) for documents are mapped to the standard HTTP methods (POST, GET, PUT, DELETE). There is also a partial update method, which is mapped to the HTTP PATCH method.

An identifier for the document revision is returned in the ETag HTTP header. If you modify a document, you can use the If-Match field to detect conflicts. The revision of a document can be checking using the HTTP method HEAD.

Address and ETag of a Document

All documents in ArangoDB have a document handle. This handle uniquely identifies a document. Any document can be retrieved using its unique URI:

http://server:port/_api/document/<document-handle>

For example, assumed that the document handle, which is stored in the _id attribute of the document, is demo/362549736, then the URL of that document is:

http://localhost:8529/_api/document/demo/362549736

The above URL scheme does not specify a database name explicitly, so the default database will be used. To explicitly specify the database context, use the following URL schema:

http://server:port/_db/<database-name>/_api/document/<document-handle>

Example:

http://localhost:8529/_db/mydb/_api/document/demo/362549736

Note: The following examples use the short URL format for brevity.

Each document also has a document revision or Etag with is returned in the "ETag" HTTP header when requesting a document.

If you obtain a document using GET and you want to check if a newer revision is available, then you can use the If-None-Match header. If the document is unchanged, a HTTP 412 (precondition failed) error is returned.

If you want to update or delete a document, then you can use the If-Match header. If the document has changed, then the operation is aborted and a HTTP 412 error is returned.

Working with Documents using REST

Read document

reads a single document

GET /_api/document/{collection-name}/{document-key}

Path Parameters

- collection-name (required): The name of the collection.

- document-key (required): The key of the document.

Header Parameters

- If-None-Match (optional): If the "If-None-Match" header is given, then it must contain exactly one etag. The document is returned, if it has a different revision than the given etag. Otherwise an HTTP 304 is returned.

- If-Match (optional): If the "If-Match" header is given, then it must contain exactly one etag. The document is returned, if it has the same revision as the given etag. Otherwise a HTTP 412 is returned. As an alternative you can supply the etag in an attribute rev in the URL.

Returns the document identified by document-handle. The returned document contains three special attributes: _id containing the document handle, _key containing key which uniquely identifies a document in a given collection and _rev containing the revision.

Return Codes

- 200: is returned if the document was found

- 304: is returned if the "If-None-Match" header is given and the document has the same version

- 404: is returned if the document or collection was not found

- 412: is returned if a "If-Match" header or rev is given and the found document has a different version. The response will also contain the found document's current revision in the _rev attribute. Additionally, the attributes _id and _key will be returned.

Examples

Use a document handle:

shell> curl --dump - http://localhost:8529/_api/document/products/731640040

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

etag: "731640040"

show response body

Use a document handle and an etag:

shell> curl --header 'If-None-Match: "735310056"' --dump - http://localhost:8529/_api/document/products/735310056

Unknown document handle:

shell> curl --dump - http://localhost:8529/_api/document/products/unknownhandle

HTTP/1.1 404 Not Found

content-type: application/json; charset=utf-8

show response body

Create document

creates a document

POST /_api/document

Post Body document (required): A JSON representation of the document.

Query Parameters

- collection (required): The collection name.

- createCollection (optional): If this parameter has a value of true or yes, then the collection is created if it does not yet exist. Other values will be ignored so the collection must be present for the operation to succeed.

Note: this flag is not supported in a cluster. Using it will result in an error.

- waitForSync (optional): Wait until document has been synced to disk.

Creates a new document in the collection named collection. A JSON representation of the document must be passed as the body of the POST request.

If the document was created successfully, then the "Location" header contains the path to the newly created document. The "ETag" header field contains the revision of the document.

The body of the response contains a JSON object with the following attributes:

- _id contains the document handle of the newly created document

- _key contains the document key

- _rev contains the document revision

If the collection parameter waitForSync is false, then the call returns as soon as the document has been accepted. It will not wait until the document has been synced to disk.

Optionally, the query parameter waitForSync can be used to force synchronization of the document creation operation to disk even in case that the waitForSync flag had been disabled for the entire collection. Thus, the waitForSync query parameter can be used to force synchronization of just this specific operations. To use this, set the waitForSync parameter to true. If the waitForSync parameter is not specified or set to false, then the collection's default waitForSync behavior is applied. The waitForSync query parameter cannot be used to disable synchronization for collections that have a default waitForSync value of true.

Return Codes

- 201: is returned if the document was created successfully and waitForSync was true.

- 202: is returned if the document was created successfully and waitForSync was false.

- 400: is returned if the body does not contain a valid JSON representation of a document. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

Create a document in a collection named products. Note that the revision identifier might or might not by equal to the auto-generated key.

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/document?collection=products <<EOF

{ "Hello": "World" }

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

etag: "730067176"

location: /_db/_system/_api/document/products/730067176

show response body

Create a document in a collection named products with a collection-level waitForSync value of false.

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/document?collection=products <<EOF

{ "Hello": "World" }

EOF

HTTP/1.1 202 Accepted

content-type: application/json; charset=utf-8

etag: "729542888"

location: /_db/_system/_api/document/products/729542888

show response body

Create a document in a collection with a collection-level waitForSync value of false, but using the waitForSync query parameter.

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/document?collection=products&waitForSync=true <<EOF

{ "Hello": "World" }

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

etag: "731115752"

location: /_db/_system/_api/document/products/731115752

show response body

Create a document in a new, named collection

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/document?collection=products&createCollection=true <<EOF

{ "Hello": "World" }

EOF

HTTP/1.1 202 Accepted

content-type: application/json; charset=utf-8

etag: "730591464"

location: /_db/_system/_api/document/products/730591464

show response body

Unknown collection name

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/document?collection=products <<EOF

{ "Hello": "World" }

EOF

HTTP/1.1 404 Not Found

content-type: application/json; charset=utf-8

show response body

Illegal document

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/document?collection=products <<EOF

{ 1: "World" }

EOF

HTTP/1.1 400 Bad Request

content-type: application/json; charset=utf-8

show response body

Replace document

replaces a document

PUT /_api/document/{collection-name}/{document-key}

Post Body document (required): A JSON representation of the new document.

Path Parameters

- collection-name (required): The name of the collection.

- document-key (required): The key of the document.

Query Parameters

- waitForSync (optional): Wait until document has been synced to disk.

- rev (optional): You can conditionally replace a document based on a target revision id by using the rev query parameter.

- policy (optional): To control the update behavior in case there is a revision mismatch, you can use the policy parameter (see below).

Header Parameters

- If-Match (optional): You can conditionally replace a document based on a target revision id by using the if-match HTTP header.

Completely updates (i.e. replaces) the document identified by document-handle. If the document exists and can be updated, then a HTTP 201 is returned and the "ETag" header field contains the new revision of the document.

If the new document passed in the body of the request contains the document-handle in the attribute _id and the revision in _rev, these attributes will be ignored. Only the URI and the "ETag" header are relevant in order to avoid confusion when using proxies.

Optionally, the query parameter waitForSync can be used to force synchronization of the document replacement operation to disk even in case that the waitForSync flag had been disabled for the entire collection. Thus, the waitForSync query parameter can be used to force synchronization of just specific operations. To use this, set the waitForSync parameter to true. If the waitForSync parameter is not specified or set to false, then the collection's default waitForSync behavior is applied. The waitForSync query parameter cannot be used to disable synchronization for collections that have a default waitForSync value of true.

The body of the response contains a JSON object with the information about the handle and the revision. The attribute _id contains the known document-handle of the updated document, _key contains the key which uniquely identifies a document in a given collection, and the attribute _rev contains the new document revision.

If the document does not exist, then a HTTP 404 is returned and the body of the response contains an error document.

There are two ways for specifying the targeted document revision id for conditional replacements (i.e. replacements that will only be executed if the revision id found in the database matches the document revision id specified in the request):

- specifying the target revision in the rev URL query parameter

- specifying the target revision in the if-match HTTP header

Specifying a target revision is optional, however, if done, only one of the described mechanisms must be used (either the rev query parameter or the if-match HTTP header). Regardless which mechanism is used, the parameter needs to contain the target document revision id as returned in the _rev attribute of a document or by an HTTP etag header.

For example, to conditionally replace a document based on a specific revision id, you can use the following request:

PUT /_api/document/collection-name/document-key?rev=etag

If a target revision id is provided in the request (e.g. via the etag value in the rev URL query parameter above), ArangoDB will check that the revision id of the document found in the database is equal to the target revision id provided in the request. If there is a mismatch between the revision id, then by default a HTTP 412 conflict is returned and no replacement is performed.

The conditional update behavior can be overridden with the policy URL query parameter:

PUT /_api/document/collection-name/document-key?policy=policy

If policy is set to error, then the behavior is as before: replacements will fail if the revision id found in the database does not match the target revision id specified in the request.

If policy is set to last, then the replacement will succeed, even if the revision id found in the database does not match the target revision id specified in the request. You can use the last policy to force replacements.

Return Codes

- 201: is returned if the document was replaced successfully and waitForSync was true.

- 202: is returned if the document was replaced successfully and waitForSync was false.

- 400: is returned if the body does not contain a valid JSON representation of a document. The response body contains an error document in this case.

- 404: is returned if the collection or the document was not found

- 412: is returned if a "If-Match" header or rev is given and the found document has a different version. The response will also contain the found document's current revision in the _rev attribute. Additionally, the attributes _id and _key will be returned.

Examples

Using a document handle

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/document/products/735899880 <<EOF

{"Hello": "you"}

EOF

HTTP/1.1 202 Accepted

content-type: application/json; charset=utf-8

etag: "736227560"

location: /_db/_system/_api/document/products/735899880

show response body

Unknown document handle

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/document/products/740094184 <<EOF

{}

EOF

HTTP/1.1 404 Not Found

content-type: application/json; charset=utf-8

show response body

Produce a revision conflict

shell> curl -X PUT --header 'If-Match: "737079528"' --data-binary @- --dump - http://localhost:8529/_api/document/products/736751848 <<EOF

{"other":"content"}

EOF

HTTP/1.1 412 Precondition Failed

content-type: application/json; charset=utf-8

etag: "736751848"

show response body

Last write wins

shell> curl -X PUT --header 'If-Match: "738193640"' --data-binary @- --dump - http://localhost:8529/_api/document/products/737865960?policy=last <<EOF

{}

EOF

HTTP/1.1 202 Accepted

content-type: application/json; charset=utf-8

etag: "738455784"

location: /_db/_system/_api/document/products/737865960

show response body

Alternative to header fields

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/document/products/738980072?rev=739307752 <<EOF

{"other":"content"}

EOF

HTTP/1.1 412 Precondition Failed

content-type: application/json; charset=utf-8

etag: "738980072"

show response body

Patch document

updates a document

PATCH /_api/document/{collection-name}/{document-key}

Post Body document (required): A JSON representation of the document update.

Path Parameters

- collection-name (required): The name of the collection.

- document-key (required): The key of the document.

Query Parameters

- keepNull (optional): If the intention is to delete existing attributes with the patch command, the URL query parameter keepNull can be used with a value of false. This will modify the behavior of the patch command to remove any attributes from the existing document that are contained in the patch document with an attribute value of null.

- mergeObjects (optional): Controls whether objects (not arrays) will be merged if present in both the existing and the patch document. If set to false, the value in the patch document will overwrite the existing document's value. If set to true, objects will be merged. The default is true.

- waitForSync (optional): Wait until document has been synced to disk.

- rev (optional): You can conditionally patch a document based on a target revision id by using the rev query parameter.

- policy (optional): To control the update behavior in case there is a revision mismatch, you can use the policy parameter.

Header Parameters

- If-Match (optional): You can conditionally patch a document based on a target revision id by using the if-match HTTP header.

Partially updates the document identified by document-handle. The body of the request must contain a JSON document with the attributes to patch (the patch document). All attributes from the patch document will be added to the existing document if they do not yet exist, and overwritten in the existing document if they do exist there.

Setting an attribute value to null in the patch document will cause a value of null be saved for the attribute by default.

Optionally, the query parameter waitForSync can be used to force synchronization of the document update operation to disk even in case that the waitForSync flag had been disabled for the entire collection. Thus, the waitForSync query parameter can be used to force synchronization of just specific operations. To use this, set the waitForSync parameter to true. If the waitForSync parameter is not specified or set to false, then the collection's default waitForSync behavior is applied. The waitForSync query parameter cannot be used to disable synchronization for collections that have a default waitForSync value of true.

The body of the response contains a JSON object with the information about the handle and the revision. The attribute _id contains the known document-handle of the updated document, _key contains the key which uniquely identifies a document in a given collection, and the attribute _rev contains the new document revision.

If the document does not exist, then a HTTP 404 is returned and the body of the response contains an error document.

You can conditionally update a document based on a target revision id by using either the rev query parameter or the if-match HTTP header. To control the update behavior in case there is a revision mismatch, you can use the policy parameter. This is the same as when replacing documents (see replacing documents for details).

Return Codes

- 201: is returned if the document was created successfully and waitForSync was true.

- 202: is returned if the document was created successfully and waitForSync was false.

- 400: is returned if the body does not contain a valid JSON representation of a document. The response body contains an error document in this case.

- 404: is returned if the collection or the document was not found

- 412: is returned if a "If-Match" header or rev is given and the found document has a different version. The response will also contain the found document's current revision in the _rev attribute. Additionally, the attributes _id and _key will be returned.

Examples

patches an existing document with new content.

shell> curl -X PATCH --data-binary @- --dump - http://localhost:8529/_api/document/products/725741800 <<EOF

{

"hello" : "world"

}

EOF

HTTP/1.1 202 Accepted

content-type: application/json; charset=utf-8

etag: "726069480"

location: /_db/_system/_api/document/products/725741800

show response body

Merging attributes of an object using  mergeObjects :

shell> curl --dump - http://localhost:8529/_api/document/products/727970024

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

etag: "727970024"

show response body

Removes a document

removes a document

DELETE /_api/document/{collection-name}/{document-key}

Path Parameters

- collection-name (required): The name of the collection.

- document-key (required): The key of the document.

Query Parameters

- rev (optional): You can conditionally remove a document based on a target revision id by using the rev query parameter.

- policy (optional): To control the update behavior in case there is a revision mismatch, you can use the policy parameter. This is the same as when replacing documents (see replacing documents for more details).

- waitForSync (optional): Wait until deletion operation has been synced to disk.

Header Parameters

- If-Match (optional): You can conditionally remove a document based on a target revision id by using the if-match HTTP header.

The body of the response contains a JSON object with the information about the handle and the revision. The attribute _id contains the known document-handle of the removed document, _key contains the key which uniquely identifies a document in a given collection, and the attribute _rev contains the new document revision.

If the waitForSync parameter is not specified or set to false, then the collection's default waitForSync behavior is applied. The waitForSync query parameter cannot be used to disable synchronization for collections that have a default waitForSync value of true.

Return Codes

- 200: is returned if the document was removed successfully and waitForSync was true.

- 202: is returned if the document was removed successfully and waitForSync was false.

- 404: is returned if the collection or the document was not found. The response body contains an error document in this case.

- 412: is returned if a "If-Match" header or rev is given and the found document has a different version. The response will also contain the found document's current revision in the _rev attribute. Additionally, the attributes _id and _key will be returned.

Examples

Using document handle:

shell> curl -X DELETE --dump - http://localhost:8529/_api/document/products/723185896

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Unknown document handle:

shell> curl -X DELETE --dump - http://localhost:8529/_api/document/products/724889832

HTTP/1.1 404 Not Found

content-type: application/json; charset=utf-8

show response body

Revision conflict:

shell> curl -X DELETE --header 'If-Match: "724234472"' --dump - http://localhost:8529/_api/document/products/723906792

HTTP/1.1 412 Precondition Failed

content-type: application/json; charset=utf-8

etag: "723906792"

show response body

Read document header

reads a single document head

HEAD /_api/document/{collection-name}/{document-key}

Path Parameters

- collection-name (required): The name of the collection.

- document-key (required): The key of the document.

Query Parameters

- rev (optional): You can conditionally fetch a document based on a target revision id by using the rev query parameter.

Header Parameters

- If-None-Match (optional): If the "If-None-Match" header is given, then it must contain exactly one etag. If the current document revision is different to the specified etag, an HTTP 200 response is returned. If the current document revision is identical to the specified etag, then an HTTP 304 is returned.

- If-Match (optional): You can conditionally fetch a document based on a target revision id by using the if-match HTTP header.

Like GET, but only returns the header fields and not the body. You can use this call to get the current revision of a document or check if the document was deleted.

Return Codes

- 200: is returned if the document was found

- 304: is returned if the "If-None-Match" header is given and the document has same version ///*

- 404: is returned if the document or collection was not found

- 412: is returned if a "If-Match" header or rev is given and the found document has a different version. The response will also contain the found document's current revision in the etag header.

Examples

shell> curl -X HEAD --dump - http://localhost:8529/_api/document/products/734720232

Read all documents

reads all documents from collection

GET /_api/document

Query Parameters

- collection (required): The name of the collection.

- type (optional): The type of the result. The following values are allowed:

- id: returns an array of document ids (_id attributes)

- key: returns an array of document keys (_key attributes)

- path: returns an array of document URI paths. This is the default.

Returns an array of all keys, ids, or URI paths for all documents in the collection identified by collection. The type of the result array is determined by the type attribute.

Note that the results have no defined order and thus the order should not be relied on.

Return Codes

- 200: All went good.

- 404: The collection does not exist.

Examples

Return all document paths

shell> curl --dump - http://localhost:8529/_api/document/?collection=products

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Return all document keys

shell> curl --dump - http://localhost:8529/_api/document/?collection=products&type=key

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Collection does not exist

shell> curl --dump - http://localhost:8529/_api/document/?collection=doesnotexist

HTTP/1.1 404 Not Found

content-type: application/json; charset=utf-8

show response body

HTTP Interface for Edges

This is an introduction to ArangoDB's REST interface for edges.

ArangoDB offers graph functionality; Edges are one part of that.

Address and ETag of an Edge

All documents in ArangoDB have a document handle. This handle uniquely identifies a document. Any document can be retrieved using its unique URI:

http://server:port/_api/document/<document-handle>

Edges are a special variation of documents, and to work with edges, the above URL format changes to:

http://server:port/_api/edge/<document-handle>

For example, assumed that the document handle, which is stored in the _id attribute of the edge, is demo/362549736, then the URL of that edge is:

http://localhost:8529/_api/edge/demo/362549736

The above URL scheme does not specify a database name explicitly, so the default database will be used. To explicitly specify the database context, use the following URL schema:

http://server:port/_db/<database-name>/_api/edge/<document-handle>

Example:

http://localhost:8529/_db/mydb/_api/edge/demo/362549736

Note: that the following examples use the short URL format for brevity.

Working with Edges using REST

This is documentation to ArangoDB's REST interface for edges.

Read edge

reads a single edge

GET /_api/edge/{collection-name}/{document-key}

Path Parameters

- collection-name (required): The name of the edge collection.

- document-key (required): The key of the edge document.

Header Parameters

- If-None-Match (optional): If the "If-None-Match" header is given, then it must contain exactly one etag. The edge is returned if it has a different revision than the given etag. Otherwise an HTTP 304 is returned.

- If-Match (optional): If the "If-Match" header is given, then it must contain exactly one etag. The edge is returned if it has the same revision ad the given etag. Otherwise a HTTP 412 is returned. As an alternative you can supply the etag in an attribute rev in the URL.

Returns the edge identified by document-handle. The returned edge contains a few special attributes:

- _id contains the document handle

- _rev contains the revision

- _from and to contain the document handles of the connected vertex documents

Return Codes

- 200: is returned if the edge was found

- 304: is returned if the "If-None-Match" header is given and the edge has the same version

- 404: is returned if the edge or collection was not found

- 412: is returned if a "If-Match" header or rev is given and the found document has a different version. The response will also contain the found document's current revision in the _rev attribute. Additionally, the attributes _id and _key will be returned.

Read all edges from collection

reads all edges from collection

GET /_api/edge

Query Parameters

- collection (required): The name of the collection.

Returns an array of all URIs for all edges from the collection identified by collection.

Return Codes

- 200: All went good.

- 404: The collection does not exist.

Create edge

creates an edge

POST /_api/edge

Post Body edge-document (required): A JSON representation of the edge document must be passed as the body of the POST request. This JSON object may contain the edge's document key in the _key attribute if needed.

Query Parameters

- collection (required): Creates a new edge in the collection identified by collection name.

- createCollection (optional): If this parameter has a value of true or yes, then the collection is created if it does not yet exist. Other values will be ignored so the collection must be present for the operation to succeed.

Note: This flag is not supported in a cluster. Using it will result in an error.

- waitForSync (optional): Wait until the edge document has been synced to disk.

- from (required): The document handle of the start point must be passed in from handle.

- to (required): The document handle of the end point must be passed in to handle.

Creates a new edge document in the collection named collection. A JSON representation of the document must be passed as the body of the POST request.

The from and to handles are immutable once the edge has been created.

In all other respects the method works like POST /document.

Return Codes

- 201: is returned if the edge was created successfully and waitForSync was true.

- 202: is returned if the edge was created successfully and waitForSync was false.

- 400: is returned if the body does not contain a valid JSON representation of an edge, or if the collection specified is not an edge collection. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

Create an edge and read it back:

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/edge/?collection=edges&from=vertices/1&to=vertices/2 <<EOF

{

"name" : "Emil"

}

EOF

HTTP/1.1 202 Accepted

content-type: application/json; charset=utf-8

etag: "742977768"

location: /_db/_system/_api/edge/edges/742977768

show response body

Patches edge

updates an edge

PATCH /_api/edge/{collection-name}/{document-key}

Post Body document (required): A JSON representation of the edge update.

Path Parameters

- collection-name (required): The name of the edge collection.

- document-key (required): The key of the edge document.

Query Parameters

- keepNull (optional): If the intention is to delete existing attributes with the patch command, the URL query parameter keepNull can be used with a value of false. This will modify the behavior of the patch command to remove any attributes from the existing edge document that are contained in the patch document with an attribute value of null.

- mergeObjects (optional): Controls whether objects (not arrays) will be merged if present in both the existing and the patch edge. If set to false, the value in the patch edge will overwrite the existing edge's value. If set to true, objects will be merged. The default is true.

- waitForSync (optional): Wait until edge document has been synced to disk.

- rev (optional): You can conditionally patch an edge document based on a target revision id by using the rev query parameter.

- policy (optional): To control the update behavior in case there is a revision mismatch, you can use the policy parameter.

Header Parameters

- If-Match (optional): You can conditionally patch an edge document based on a target revision id by using the if-match HTTP header.

Partially updates the edge document identified by document-handle. The body of the request must contain a JSON document with the attributes to patch (the patch document). All attributes from the patch document will be added to the existing edge document if they do not yet exist, and overwritten in the existing edge document if they do exist there.

Setting an attribute value to null in the patch document will cause a value of null be saved for the attribute by default.

Note: Internal attributes such as _key, _from and _to are immutable once set and cannot be updated.

Optionally, the query parameter waitForSync can be used to force synchronization of the edge document update operation to disk even in case that the waitForSync flag had been disabled for the entire collection. Thus, the waitForSync query parameter can be used to force synchronization of just specific operations. To use this, set the waitForSync parameter to true. If the waitForSync parameter is not specified or set to false, then the collection's default waitForSync behavior is applied. The waitForSync query parameter cannot be used to disable synchronization for collections that have a default waitForSync value of true.

The body of the response contains a JSON object with the information about the handle and the revision. The attribute _id contains the known document-handle of the updated edge document, _key contains the key which uniquely identifies a document in a given collection, and the attribute _rev contains the new document revision.

If the edge document does not exist, then a HTTP 404 is returned and the body of the response contains an error document.

You can conditionally update an edge document based on a target revision id by using either the rev query parameter or the if-match HTTP header. To control the update behavior in case there is a revision mismatch, you can use the policy parameter. This is the same as when replacing edge documents (see replacing documents for details).

Return Codes

- 201: is returned if the document was patched successfully and waitForSync was true.

- 202: is returned if the document was patched successfully and waitForSync was false.

- 400: is returned if the body does not contain a valid JSON representation or when applied on an non-edge collection. The response body contains an error document in this case.

- 404: is returned if the collection or the edge document was not found

- 412: is returned if a "If-Match" header or rev is given and the found document has a different version. The response will also contain the found document's current revision in the _rev attribute. Additionally, the attributes _id and _key will be returned.

replaces an edge

replaces an edge

PUT /_api/edge/{collection-name}/{document-key}

Post Body edge (required): A JSON representation of the new edge data.

Path Parameters

- collection-name (required): The name of the edge collection.

- document-key (required): The key of the edge document.

Query Parameters

- waitForSync (optional): Wait until edge document has been synced to disk.

- rev (optional): You can conditionally replace an edge document based on a target revision id by using the rev query parameter.

- policy (optional): To control the update behavior in case there is a revision mismatch, you can use the policy parameter (see below).

Header Parameters

- If-Match (optional): You can conditionally replace an edge document based on a target revision id by using the if-match HTTP header.

Completely updates (i.e. replaces) the edge document identified by document-handle. If the edge document exists and can be updated, then a HTTP 201 is returned and the "ETag" header field contains the new revision of the edge document.

If the new edge document passed in the body of the request contains the document-handle in the attribute _id and the revision in _rev, these attributes will be ignored. Only the URI and the "ETag" header are relevant in order to avoid confusion when using proxies. Note: The attributes _from and _to of an edge are immutable and cannot be updated either.

Optionally, the query parameter waitForSync can be used to force synchronization of the edge document replacement operation to disk even in case that the waitForSync flag had been disabled for the entire collection. Thus, the waitForSync query parameter can be used to force synchronization of just specific operations. To use this, set the waitForSync parameter to true. If the waitForSync parameter is not specified or set to false, then the collection's default waitForSync behavior is applied. The waitForSync query parameter cannot be used to disable synchronization for collections that have a default waitForSync value of true.

The body of the response contains a JSON object with the information about the handle and the revision. The attribute _id contains the known document-handle of the updated edge document, _key contains the key which uniquely identifies a document in a given collection, and the attribute _rev contains the new document revision.

If the edge document does not exist, then a HTTP 404 is returned and the body of the response contains an error document.

There are two ways for specifying the targeted revision id for conditional replacements (i.e. replacements that will only be executed if the revision id found in the database matches the revision id specified in the request):

- specifying the target revision in the rev URL query parameter

- specifying the target revision in the if-match HTTP header

Specifying a target revision is optional, however, if done, only one of the described mechanisms must be used (either the rev query parameter or the if-match HTTP header). Regardless which mechanism is used, the parameter needs to contain the target revision id as returned in the _rev attribute of an edge document or by an HTTP etag header.

For example, to conditionally replace an edge document based on a specific revision id, you can use the following request:

- PUT /_api/document/collection-name/document-key?rev=etag

If a target revision id is provided in the request (e.g. via the etag value in the rev URL query parameter above), ArangoDB will check that the revision id of the edge document found in the database is equal to the target revision id provided in the request. If there is a mismatch between the revision id, then by default a HTTP 412 conflict is returned and no replacement is performed.

The conditional update behavior can be overridden with the policy URL query parameter:

- PUT /_api/document/collection-name/document-key?policy=policy

If policy is set to error, then the behavior is as before: replacements will fail if the revision id found in the database does not match the target revision id specified in the request.

If policy is set to last, then the replacement will succeed, even if the revision id found in the database does not match the target revision id specified in the request. You can use the last policy to force replacements.

Return Codes

- 201: is returned if the edge document was replaced successfully and waitForSync was true.

- 202: is returned if the edge document was replaced successfully and waitForSync was false.

- 400: is returned if the body does not contain a valid JSON representation of an edge document or if applied to a non-edge collection. The response body contains an error document in this case.

- 404: is returned if the collection or the edge document was not found

- 412: is returned if a "If-Match" header or rev is given and the found document has a different version. The response will also contain the found document's current revision in the _rev attribute. Additionally, the attributes _id and _key will be returned.

Deletes edge

deletes an edge

DELETE /_api/edge/{collection-name}/{document-key}

Path Parameters

- collection-name (required): The name of the edge collection.

- document-key (required): The key of the edge document.

Query Parameters

- rev (optional): You can conditionally delete an edge document based on a target revision id by using the rev query parameter.

- policy (optional): To control the update behavior in case there is a revision mismatch, you can use the policy parameter. This is the same as when replacing edge documents (see replacing edge documents for more details).

- waitForSync (optional): Wait until edge document has been synced to disk.

Header Parameters

- If-Match (optional): You can conditionally delete an edge document based on a target revision id by using the if-match HTTP header.

The body of the response contains a JSON object with the information about the handle and the revision. The attribute _id contains the known document-handle of the deleted edge document, _key contains the key which uniquely identifies a document in a given collection, and the attribute _rev contains the new document revision.

If the waitForSync parameter is not specified or set to false, then the collection's default waitForSync behavior is applied. The waitForSync query parameter cannot be used to disable synchronization for collections that have a default waitForSync value of true.

Return Codes

- 200: is returned if the edge document was deleted successfully and waitForSync was true.

- 202: is returned if the edge document was deleted successfully and waitForSync was false.

- 404: is returned if the collection or the edge document was not found. The response body contains an error document in this case.

- 412: is returned if a "If-Match" header or rev is given and the found document has a different version. The response will also contain the found document's current revision in the _rev attribute. Additionally, the attributes _id and _key will be returned.

Read edge header

reads a single edge head

HEAD /_api/edge/{collection-name}/{document-key}

Path Parameters

- collection-name (required): The name of the edge collection.

- document-key (required): The key of the edge document.

Query Parameters

- rev (optional): You can conditionally fetch an edge document based on a target revision id by using the rev query parameter.

Header Parameters

- If-None-Match (optional): If the "If-None-Match" header is given, then it must contain exactly one etag. If the current document revision is different to the specified etag, an HTTP 200 response is returned. If the current document revision is identical to the specified etag, then an HTTP 304 is returned.

- If-Match (optional): You can conditionally fetch an edge document based on a target revision id by using the if-match HTTP header.

Like GET, but only returns the header fields and not the body. You can use this call to get the current revision of an edge document or check if it was deleted.

Return Codes

- 200: is returned if the edge document was found

- 304: is returned if the "If-None-Match" header is given and the edge document has same version

- 404: is returned if the edge document or collection was not found

- 412: is returned if a "If-Match" header or rev is given and the found document has a different version. The response will also contain the found document's current revision in the etag header.

Read in- or outbound edges

get edges

GET /_api/edges/{collection-id}

Path Parameters

- collection-id (required): The id of the collection.

Query Parameters

- vertex (required): The id of the start vertex.

- direction (optional): Selects in or out direction for edges. If not set, any edges are returned.

Returns an array of edges starting or ending in the vertex identified by vertex-handle.

Return Codes

- 200: is returned if the edge collection was found and edges were retrieved.

- 400: is returned if the request contains invalid parameters.

- 404: is returned if the edge collection was not found.

Examples

Any direction

shell> curl --dump - http://localhost:8529/_api/edges/edges?vertex=vertices/1

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

In edges

shell> curl --dump - http://localhost:8529/_api/edges/edges?vertex=vertices/1&direction=in

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Out edges

shell> curl --dump - http://localhost:8529/_api/edges/edges?vertex=vertices/1&direction=out

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

HTTP Interface for Exporting Documents

Create export cursor

export all documents from a collection, using a cursor

POST /_api/export

Body Parameters

- count: boolean flag that indicates whether the number of documents in the result set should be returned in the "count" attribute of the result (optional). Calculating the "count" attribute might in the future have a performance impact so this option is turned off by default, and "count" is only returned when requested.

- restrict:

- fields: Contains an array of attribute names to include or exclude. Matching of attribute names for inclusion or exclusion will be done on the top level only. Specifying names of nested attributes is not supported at the moment. of type string

- type: has to be be set to either include or exclude depending on which you want to use

- batchSize: maximum number of result documents to be transferred from the server to the client in one roundtrip (optional). If this attribute is not set, a server-controlled default value will be used.

- ttl: an optional time-to-live for the cursor (in seconds). The cursor will be removed on the server automatically after the specified amount of time. This is useful to ensure garbage collection of cursors that are not fully fetched by clients. If not set, a server-defined value will be used.

- flushWait: maximum wait time in seconds after a flush operation. The default value is 10. This option only has an effect when flush is set to true.

- limit: an optional limit value, determining the maximum number of documents to be included in the cursor. Omitting the limit attribute or setting it to 0 will lead to no limit being used. If a limit is used, it is undefined which documents from the collection will be included in the export and which will be excluded. This is because there is no natural order of documents in a collection.

- flush: if set to true, a WAL flush operation will be executed prior to the export. The flush operation will start copying documents from the WAL to the collection's datafiles. There will be an additional wait time of up to flushWait seconds after the flush to allow the WAL collector to change the adjusted document meta-data to point into the datafiles, too. The default value is false (i.e. no flush) so most recently inserted or updated documents from the collection might be missing in the export.

Not specifying restrict will by default return all attributes of each document.

Query Parameters

- collection (required): The name of the collection to export.

A call to this method creates a cursor containing all documents in the specified collection. In contrast to other data-producing APIs, the internal data structures produced by the export API are more lightweight, so it is the preferred way to retrieve all documents from a collection.

Documents are returned in a similar manner as in the  /_api/cursor  REST API. If all documents of the collection fit into the first batch, then no cursor will be created, and the result object's hasMore attribute will be set to false. If not all documents fit into the first batch, then the result object's hasMore attribute will be set to true, and the id attribute of the result will contain a cursor id.

The order in which the documents are returned is not specified.

By default, only those documents from the collection will be returned that are stored in the collection's datafiles. Documents that are present in the write-ahead log (WAL) at the time the export is run will not be exported.

To export these documents as well, the caller can issue a WAL flush request before calling the export API or set the flush attribute. Setting the flush option will trigger a WAL flush before the export so documents get copied from the WAL to the collection datafiles.

If the result set can be created by the server, the server will respond with HTTP 201. The body of the response will contain a JSON object with the result set.

The returned JSON object has the following properties:

- error: boolean flag to indicate that an error occurred (false in this case)

- code: the HTTP status code

- result: an array of result documents (might be empty if the collection was empty)

- hasMore: a boolean indicator whether there are more results available for the cursor on the server

- count: the total number of result documents available (only available if the query was executed with the count attribute set)

- id: id of temporary cursor created on the server (optional, see above)

If the JSON representation is malformed or the query specification is missing from the request, the server will respond with HTTP 400.

The body of the response will contain a JSON object with additional error details. The object has the following attributes:

- error: boolean flag to indicate that an error occurred (true in this case)

- code: the HTTP status code

- errorNum: the server error number

- errorMessage: a descriptive error message

Clients should always delete an export cursor result as early as possible because a lingering export cursor will prevent the underlying collection from being compacted or unloaded. By default, unused cursors will be deleted automatically after a server-defined idle time, and clients can adjust this idle time by setting the ttl value.

Note: this API is currently not supported on cluster coordinators.

Return Codes

- 201: is returned if the result set can be created by the server.

- 400: is returned if the JSON representation is malformed or the query specification is missing from the request.

- 404: The server will respond with HTTP 404 in case a non-existing collection is accessed in the query.

- 405: The server will respond with HTTP 405 if an unsupported HTTP method is used.

- 501: The server will respond with HTTP 501 if this API is called on a cluster coordinator.

HTTP Interface for AQL Query Cursors

Database Cursors

This is an introduction to ArangoDB's HTTP Interface for Queries. Results of AQL and simple queries are returned as cursors in order to batch the communication between server and client. Each call returns a number of documents in a batch and an indication if the current batch has been the final batch. Depending on the query, the total number of documents in the result set might or might not be known in advance. In order to free server resources the client should delete the cursor as soon as it is no longer needed.

To execute a query, the query details need to be shipped from the client to the server via an HTTP POST request.

Retrieving query results

Select queries are executed on-the-fly on the server and the result set will be returned back to the client.

There are two ways the client can get the result set from the server:

- In a single roundtrip

- Using a cursor

Single roundtrip

The server will only transfer a certain number of result documents back to the client in one roundtrip. This number is controllable by the client by setting the batchSize attribute when issuing the query.

If the complete result can be transferred to the client in one go, the client does not need to issue any further request. The client can check whether it has retrieved the complete result set by checking the hasMore attribute of the result set. If it is set to false, then the client has fetched the complete result set from the server. In this case no server side cursor will be created.

> curl --data @- -X POST --dump - http://localhost:8529/_api/cursor

{ "query" : "FOR u IN users LIMIT 2 RETURN u", "count" : true, "batchSize" : 2 }

HTTP/1.1 201 Created

Content-type: application/json

{

"hasMore" : false,

"error" : false,

"result" : [

{

"name" : "user1",

"_rev" : "210304551",

"_key" : "210304551",

"_id" : "users/210304551"

},

{

"name" : "user2",

"_rev" : "210304552",

"_key" : "210304552",

"_id" : "users/210304552"

}

],

"code" : 201,

"count" : 2

}

Using a cursor

If the result set contains more documents than should be transferred in a single roundtrip (i.e. as set via the batchSize attribute), the server will return the first few documents and create a temporary cursor. The cursor identifier will also be returned to the client. The server will put the cursor identifier in the id attribute of the response object. Furthermore, the hasMore attribute of the response object will be set to true. This is an indication for the client that there are additional results to fetch from the server.

Examples:

Create and extract first batch:

> curl --data @- -X POST --dump - http://localhost:8529/_api/cursor

{ "query" : "FOR u IN users LIMIT 5 RETURN u", "count" : true, "batchSize" : 2 }

HTTP/1.1 201 Created

Content-type: application/json

{

"hasMore" : true,

"error" : false,

"id" : "26011191",

"result" : [

{

"name" : "user1",

"_rev" : "258801191",

"_key" : "258801191",

"_id" : "users/258801191"

},

{

"name" : "user2",

"_rev" : "258801192",

"_key" : "258801192",

"_id" : "users/258801192"

}

],

"code" : 201,

"count" : 5

}

Extract next batch, still have more:

> curl -X PUT --dump - http://localhost:8529/_api/cursor/26011191

HTTP/1.1 200 OK

Content-type: application/json

{

"hasMore" : true,

"error" : false,

"id" : "26011191",

"result": [

{

"name" : "user3",

"_rev" : "258801193",

"_key" : "258801193",

"_id" : "users/258801193"

},

{

"name" : "user4",

"_rev" : "258801194",

"_key" : "258801194",

"_id" : "users/258801194"

}

],

"code" : 200,

"count" : 5

}

Extract next batch, done:

> curl -X PUT --dump - http://localhost:8529/_api/cursor/26011191

HTTP/1.1 200 OK

Content-type: application/json

{

"hasMore" : false,

"error" : false,

"result" : [

{

"name" : "user5",

"_rev" : "258801195",

"_key" : "258801195",

"_id" : "users/258801195"

}

],

"code" : 200,

"count" : 5

}

Do not do this because hasMore now has a value of false:

> curl -X PUT --dump - http://localhost:8529/_api/cursor/26011191

HTTP/1.1 404 Not Found

Content-type: application/json

{

"errorNum": 1600,

"errorMessage": "cursor not found: disposed or unknown cursor",

"error": true,

"code": 404

}

Modifying documents

The  _api/cursor  endpoint can also be used to execute modifying queries.

The following example appends a value into the array  arrayValue  of the document with key  test  in the collection  documents . Normal update behavior is to replace the attribute completely, and using an update AQL query with the  PUSH()  function allows to append to the array.

curl --data @- -X POST --dump http://127.0.0.1:8529/_api/cursor

{ "query": "FOR doc IN documents FILTER doc._key == @myKey UPDATE doc._key WITH { arrayValue: PUSH(doc.arrayValue, @value) } IN documents","bindVars": { "myKey": "test", "value": 42 } }

HTTP/1.1 201 Created

Content-type: application/json; charset=utf-8

{

"result" : [],

"hasMore" : false,

"extra" : {

"stats" : {

"writesExecuted" : 1,

"writesIgnored" : 0,

"scannedFull" : 0,

"scannedIndex" : 1,

"filtered" : 0

},

"warnings" : []

},

"error" : false,

"code" : 201

}

Accessing Cursors via HTTP

Create cursor

create a cursor and return the first results

POST /_api/cursor

A JSON object describing the query and query parameters.

Body Parameters

- count: indicates whether the number of documents in the result set should be returned in the "count" attribute of the result. Calculating the "count" attribute might in the future have a performance impact for some queries so this option is turned off by default, and "count" is only returned when requested.

- ttl: The time-to-live for the cursor (in seconds). The cursor will be removed on the server automatically after the specified amount of time. This is useful to ensure garbage collection of cursors that are not fully fetched by clients. If not set, a server-defined value will be used.

- batchSize: maximum number of result documents to be transferred from the server to the client in one roundtrip. If this attribute is not set, a server-controlled default value will be used. A batchSize value of 0 is disallowed.

- cache: flag to determine whether the AQL query cache shall be used. If set to false, then any query cache lookup will be skipped for the query. If set to true, it will lead to the query cache being checked for the query if the query cache mode is either on or demand.

- bindVars: list of bind parameter objects. of type object

- query: contains the query string to be executed

- options:

- profile: if set to true, then the additional query profiling information will be returned in the extra.stats return attribute if the query result is not served from the query cache.

- optimizer.rules: a list of to-be-included or to-be-excluded optimizer rules can be put into this attribute, telling the optimizer to include or exclude specific rules. To disable a rule, prefix its name with a  - , to enable a rule, prefix it with a  + . There is also a pseudo-rule  all , which will match all optimizer rules. of type string

- fullCount: if set to true and the query contains a LIMIT clause, then the result will contain an extra attribute extra with a sub-attribute fullCount. This sub-attribute will contain the number of documents in the result before the last LIMIT in the query was applied. It can be used to count the number of documents that match certain filter criteria, but only return a subset of them, in one go. It is thus similar to MySQL's SQL_CALC_FOUND_ROWS hint. Note that setting the option will disable a few LIMIT optimizations and may lead to more documents being processed, and thus make queries run longer. Note that the fullCount sub-attribute will only be present in the result if the query has a LIMIT clause and the LIMIT clause is actually used in the query.

- maxPlans: limits the maximum number of plans that are created by the AQL query optimizer.

The query details include the query string plus optional query options and bind parameters. These values need to be passed in a JSON representation in the body of the POST request.

Return Codes

- 201: is returned if the result set can be created by the server.

Reply Body

- count: the total number of result documents available (only available if the query was executed with the count attribute set)

- code: the HTTP status code

- extra: an optional JSON object with extra information about the query result contained in its stats sub-attribute. For data-modification queries, the extra.stats sub-attribute will contain the number of modified documents and the number of documents that could not be modified due to an error (if ignoreErrors query option is specified)

- cached: a boolean flag indicating whether the query result was served from the query cache or not. If the query result is served from the query cache, the extra return attribute will not contain any stats sub-attribute and no profile sub-attribute.

- hasMore: A boolean indicator whether there are more results available for the cursor on the server

- result: an array of result documents (might be empty if query has no results)anonymous json object

- error: A flag to indicate that an error occurred (false in this case)

- id: id of temporary cursor created on the server (optional, see above)

- 400: is returned if the JSON representation is malformed or the query specification is missing from the request.

If the JSON representation is malformed or the query specification is missing from the request, the server will respond with HTTP 400.

The body of the response will contain a JSON object with additional error details. The object has the following attributes:

Reply Body

- errorMessage: a descriptive error message If the query specification is complete, the server will process the query. If an error occurs during query processing, the server will respond with HTTP 400. Again, the body of the response will contain details about the error. A list of query errors can be found here.

- code: the HTTP status code

- errorNum: the server error number

- error: boolean flag to indicate that an error occurred (true in this case)

If the query specification is complete, the server will process the query. If an error occurs during query processing, the server will respond with HTTP 400. Again, the body of the response will contain details about the error.

A list of query errors can be found here.

- 404: The server will respond with HTTP 404 in case a non-existing collection is accessed in the query.

- 405: The server will respond with HTTP 405 if an unsupported HTTP method is used.

Examples

Execute a query and extract the result in a single go

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF

{

"query" : "FOR p IN products LIMIT 2 RETURN p",

"count" : true,

"batchSize" : 2

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Execute a query and extract a part of the result

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF

{

"query" : "FOR p IN products LIMIT 5 RETURN p",

"count" : true,

"batchSize" : 2

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Using the query option "fullCount"

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF

{

"query" : "FOR i IN 1..1000 FILTER i > 500 LIMIT 10 RETURN i",

"count" : true,

"options" : {

"fullCount" : true

}

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Enabling and disabling optimizer rules

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF

{

"query" : "FOR i IN 1..10 LET a = 1 LET b = 2 FILTER a + b == 3 RETURN i",

"count" : true,

"options" : {

"maxPlans" : 1,

"optimizer" : {

"rules" : [

"-all",

"+remove-unnecessary-filters"

]

}

}

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Execute a data-modification query and retrieve the number of modified documents

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF

{

"query" : "FOR p IN products REMOVE p IN products"

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Execute a data-modification query with option ignoreErrors

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF

{

"query" : "REMOVE 'bar' IN products OPTIONS { ignoreErrors: true }"

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Bad query - Missing body

shell> curl -X POST --dump - http://localhost:8529/_api/cursor

HTTP/1.1 400 Bad Request

content-type: application/json; charset=utf-8

show response body

Bad query - Unknown collection

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF

{

"query" : "FOR u IN unknowncoll LIMIT 2 RETURN u",

"count" : true,

"batchSize" : 2

}

EOF

HTTP/1.1 404 Not Found

content-type: application/json; charset=utf-8

show response body

Bad query - Execute a data-modification query that attempts to remove a non-existing document

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF

{

"query" : "REMOVE 'foo' IN products"

}

EOF

HTTP/1.1 404 Not Found

content-type: application/json; charset=utf-8

show response body

Read next batch from cursor

return the next results from an existing cursor

PUT /_api/cursor/{cursor-identifier}

Path Parameters

- cursor-identifier (required): The name of the cursor

If the cursor is still alive, returns an object with the following attributes:

- id: the cursor-identifier

- result: a list of documents for the current batch

- hasMore: false if this was the last batch

- count: if present the total number of elements

Note that even if hasMore returns true, the next call might still return no documents. If, however, hasMore is false, then the cursor is exhausted. Once the hasMore attribute has a value of false, the client can stop.

Return Codes

- 200: The server will respond with HTTP 200 in case of success.

- 400: If the cursor identifier is omitted, the server will respond with HTTP 404.

- 404: If no cursor with the specified identifier can be found, the server will respond with HTTP 404.

Examples

Valid request for next batch

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF

{

"query" : "FOR p IN products LIMIT 5 RETURN p",

"count" : true,

"batchSize" : 2

}

EOF

shell> curl -X PUT --dump - http://localhost:8529/_api/cursor/678949096

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Missing identifier

shell> curl -X PUT --dump - http://localhost:8529/_api/cursor

HTTP/1.1 400 Bad Request

content-type: application/json; charset=utf-8

show response body

Unknown identifier

shell> curl -X PUT --dump - http://localhost:8529/_api/cursor/123123

HTTP/1.1 404 Not Found

content-type: application/json; charset=utf-8

show response body

Delete cursor

dispose an existing cursor

DELETE /_api/cursor/{cursor-identifier}

Path Parameters

- cursor-identifier (required): The id of the cursor

Deletes the cursor and frees the resources associated with it.

The cursor will automatically be destroyed on the server when the client has retrieved all documents from it. The client can also explicitly destroy the cursor at any earlier time using an HTTP DELETE request. The cursor id must be included as part of the URL.

Note: the server will also destroy abandoned cursors automatically after a certain server-controlled timeout to avoid resource leakage.

Return Codes

- 202: is returned if the server is aware of the cursor.

- 404: is returned if the server is not aware of the cursor. It is also returned if a cursor is used after it has been destroyed.

Examples

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF

{

"query" : "FOR p IN products LIMIT 5 RETURN p",

"count" : true,

"batchSize" : 2

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

HTTP Interface for AQL Queries

Explaining and parsing queries

ArangoDB has an HTTP interface to syntactically validate AQL queries. Furthermore, it offers an HTTP interface to retrieve the execution plan for any valid AQL query.

Both functionalities do not actually execute the supplied AQL query, but only inspect it and return meta information about it.

Explain an AQL query

explain an AQL query and return information about it

POST /_api/explain

A JSON object describing the query and query parameters.

Body Parameters

- query: the query which you want explained; If the query references any bind variables, these must also be passed in the attribute bindVars. Additional options for the query can be passed in the options attribute.

- options:

- optimizer.rules: an array of to-be-included or to-be-excluded optimizer rules can be put into this attribute, telling the optimizer to include or exclude specific rules. To disable a rule, prefix its name with a  - , to enable a rule, prefix it with a  + . There is also a pseudo-rule  all , which will match all optimizer rules. of type string

- maxNumberOfPlans: an optional maximum number of plans that the optimizer is allowed to generate. Setting this attribute to a low value allows to put a cap on the amount of work the optimizer does.

- allPlans: if set to true, all possible execution plans will be returned. The default is false, meaning only the optimal plan will be returned.

- bindVars: key/value pairs representing the bind values of type object

To explain how an AQL query would be executed on the server, the query string can be sent to the server via an HTTP POST request. The server will then validate the query and create an execution plan for it. The execution plan will be returned, but the query will not be executed.

The execution plan that is returned by the server can be used to estimate the probable performance of the query. Though the actual performance will depend on many different factors, the execution plan normally can provide some rough estimates on the amount of work the server needs to do in order to actually run the query.

By default, the explain operation will return the optimal plan as chosen by the query optimizer The optimal plan is the plan with the lowest total estimated cost. The plan will be returned in the attribute plan of the response object. If the option allPlans is specified in the request, the result will contain all plans created by the optimizer. The plans will then be returned in the attribute plans.

The result will also contain an attribute warnings, which is an array of warnings that occurred during optimization or execution plan creation. Additionally, a stats attribute is contained in the result with some optimizer statistics.

Each plan in the result is a JSON object with the following attributes:

- nodes: the array of execution nodes of the plan. The array of available node types can be found here

- estimatedCost: the total estimated cost for the plan. If there are multiple plans, the optimizer will choose the plan with the lowest total cost.

- collections: an array of collections used in the query

- rules: an array of rules the optimizer applied. An overview of the available rules can be found here

- variables: array of variables used in the query (note: this may contain internal variables created by the optimizer)

Return Codes

- 200: If the query is valid, the server will respond with HTTP 200 and return the optimal execution plan in the plan attribute of the response. If option allPlans was set in the request, an array of plans will be returned in the allPlans attribute instead.

- 400: The server will respond with HTTP 400 in case of a malformed request, or if the query contains a parse error. The body of the response will contain the error details embedded in a JSON object. Omitting bind variables if the query references any will also result in an HTTP 400 error.

- 404: The server will respond with HTTP 404 in case a non-existing collection is accessed in the query.

Examples

Valid query

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/explain <<EOF

{

"query" : "FOR p IN products RETURN p"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

A plan with some optimizer rules applied

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/explain <<EOF

{

"query" : "FOR p IN products LET a = p.id FILTER a == 4 LET name = p.name SORT p.id LIMIT 1 RETURN name"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Using some options

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/explain <<EOF

{

"query" : "FOR p IN products LET a = p.id FILTER a == 4 LET name = p.name SORT p.id LIMIT 1 RETURN name",

"options" : {

"maxNumberOfPlans" : 2,

"allPlans" : true,

"optimizer" : {

"rules" : [

"-all",

"+use-index-for-sort",

"+use-index-range"

]

}

}

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Returning all plans

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/explain <<EOF

{

"query" : "FOR p IN products FILTER p.id == 25 RETURN p",

"options" : {

"allPlans" : true

}

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

A query that produces a warning

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/explain <<EOF

{

"query" : "FOR i IN 1..10 RETURN 1 / 0"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Invalid query (missing bind parameter)

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/explain <<EOF

{

"query" : "FOR p IN products FILTER p.id == @id LIMIT 2 RETURN p.n"

}

EOF

HTTP/1.1 400 Bad Request

content-type: application/json; charset=utf-8

show response body

The data returned in the plan attribute of the result contains one element per AQL top-level statement (i.e. FOR, RETURN, FILTER etc.). If the query optimizer removed some unnecessary statements, the result might also contain less elements than there were top-level statements in the AQL query. The following example shows a query with a non-sensible filter condition that the optimizer has removed so that there are less top-level statements.

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/explain <<EOF

{ "query" : "FOR i IN [ 1, 2, 3 ] FILTER 1 == 2 RETURN i" }

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Parse an AQL query

parse an AQL query and return information about it

POST /_api/query

Body Parameters

- query: To validate a query string without executing it, the query string can be passed to the server via an HTTP POST request.

Return Codes

- 200: If the query is valid, the server will respond with HTTP 200 and return the names of the bind parameters it found in the query (if any) in the bindVars attribute of the response. It will also return an array of the collections used in the query in the collections attribute. If a query can be parsed successfully, the ast attribute of the returned JSON will contain the abstract syntax tree representation of the query. The format of the ast is subject to change in future versions of ArangoDB, but it can be used to inspect how ArangoDB interprets a given query. Note that the abstract syntax tree will be returned without any optimizations applied to it.

- 400: The server will respond with HTTP 400 in case of a malformed request, or if the query contains a parse error. The body of the response will contain the error details embedded in a JSON object.

Examples

a Valid query

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/query <<EOF

{ "query" : "FOR p IN products FILTER p.name == @name LIMIT 2 RETURN p.n" }

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

an Invalid query

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/query <<EOF

{ "query" : "FOR p IN products FILTER p.name = @name LIMIT 2 RETURN p.n" }

EOF

HTTP/1.1 400 Bad Request

content-type: application/json; charset=utf-8

show response body

Query tracking

ArangoDB has an HTTP interface for retrieving the lists of currently executing AQL queries and the list of slow AQL queries. In order to make meaningful use of these APIs, query tracking needs to be enabled in the database the HTTP request is executed for.

Returns the properties for the AQL query tracking

returns the configuration for the AQL query tracking

GET /_api/query/properties

Returns the current query tracking configuration. The configuration is a JSON object with the following properties:

- enabled: if set to true, then queries will be tracked. If set to false, neither queries nor slow queries will be tracked.

- trackSlowQueries: if set to true, then slow queries will be tracked in the list of slow queries if their runtime exceeds the value set in slowQueryThreshold. In order for slow queries to be tracked, the enabled property must also be set to true.

- maxSlowQueries: the maximum number of slow queries to keep in the list of slow queries. If the list of slow queries is full, the oldest entry in it will be discarded when additional slow queries occur.

- slowQueryThreshold: the threshold value for treating a query as slow. A query with a runtime greater or equal to this threshold value will be put into the list of slow queries when slow query tracking is enabled. The value for slowQueryThreshold is specified in seconds.

- maxQueryStringLength: the maximum query string length to keep in the list of queries. Query strings can have arbitrary lengths, and this property can be used to save memory in case very long query strings are used. The value is specified in bytes.

Return Codes

- 200: Is returned if properties were retrieved successfully.

- 400: The server will respond with HTTP 400 in case of a malformed request,

Changes the properties for the AQL query tracking

changes the configuration for the AQL query tracking

PUT /_api/query/properties

Body Parameters

- slowQueryThreshold: The threshold value for treating a query as slow. A query with a runtime greater or equal to this threshold value will be put into the list of slow queries when slow query tracking is enabled. The value for slowQueryThreshold is specified in seconds.

- enabled: If set to true, then queries will be tracked. If set to false, neither queries nor slow queries will be tracked.

- maxQueryStringLength: The maximum query string length to keep in the list of queries. Query strings can have arbitrary lengths, and this property can be used to save memory in case very long query strings are used. The value is specified in bytes.

- trackSlowQueries: If set to true, then slow queries will be tracked in the list of slow queries if their runtime exceeds the value set in slowQueryThreshold. In order for slow queries to be tracked, the enabled property must also be set to true.

- maxSlowQueries: The maximum number of slow queries to keep in the list of slow queries. If the list of slow queries is full, the oldest entry in it will be discarded when additional slow queries occur.

The properties need to be passed in the attribute properties in the body of the HTTP request. properties needs to be a JSON object.

After the properties have been changed, the current set of properties will be returned in the HTTP response.

Return Codes

- 200: Is returned if the properties were changed successfully.

- 400: The server will respond with HTTP 400 in case of a malformed request,

Returns the currently running AQL queries

returns a list of currently running AQL queries

GET /_api/query/current

Returns an array containing the AQL queries currently running in the selected database. Each query is a JSON object with the following attributes:

- id: the query's id

- query: the query string (potentially truncated)

- started: the date and time when the query was started

- runTime: the query's run time up to the point the list of queries was queried

Return Codes

- 200: Is returned when the list of queries can be retrieved successfully.

- 400: The server will respond with HTTP 400 in case of a malformed request,

Returns the list of slow AQL queries

returns a list of slow running AQL queries

GET /_api/query/slow

Returns an array containing the last AQL queries that exceeded the slow query threshold in the selected database. The maximum amount of queries in the list can be controlled by setting the query tracking property  maxSlowQueries . The threshold for treating a query as slow can be adjusted by setting the query tracking property  slowQueryThreshold .

Each query is a JSON object with the following attributes:

- id: the query's id

- query: the query string (potentially truncated)

- started: the date and time when the query was started

- runTime: the query's run time up to the point the list of queries was queried

Return Codes

- 200: Is returned when the list of queries can be retrieved successfully.

- 400: The server will respond with HTTP 400 in case of a malformed request,

Clears the list of slow AQL queries

clears the list of slow AQL queries

DELETE /_api/query/slow

Clears the list of slow AQL queries

Return Codes

- 200: The server will respond with HTTP 200 when the list of queries was cleared successfully.

- 400: The server will respond with HTTP 400 in case of a malformed request.

Killing queries

Running AQL queries can also be killed on the server. ArangoDB provides a kill facility via an HTTP interface. To kill a running query, its id (as returned for the query in the list of currently running queries) must be specified. The kill flag of the query will then be set, and the query will be aborted as soon as it reaches a cancelation point.

Kills a running AQL query

kills an AQL query

DELETE /_api/query/{query-id}

Path Parameters

- query-id (required): The id of the query.

Kills a running query. The query will be terminated at the next cancelation point.

Return Codes

- 200: The server will respond with HTTP 200 when the query was still running when the kill request was executed and the query's kill flag was set.

- 400: The server will respond with HTTP 400 in case of a malformed request.

- 404: The server will respond with HTTP 404 when no query with the specified id was found.

HTTP Interface for the AQL query cache

This section describes the API methods for controlling the AQL query cache.

Clears any results in the AQL query cache

clears the AQL query cache

DELETE /_api/query-cache

clears the query cache

Return Codes

- 200: The server will respond with HTTP 200 when the cache was cleared successfully.

- 400: The server will respond with HTTP 400 in case of a malformed request.

Returns the global properties for the AQL query cache

returns the global configuration for the AQL query cache

GET /_api/query-cache/properties

Returns the global AQL query cache configuration. The configuration is a JSON object with the following properties:

- mode: the mode the AQL query cache operates in. The mode is one of the following values: off, on or demand.

- maxResults: the maximum number of query results that will be stored per database-specific cache.

Return Codes

- 200: Is returned if the properties can be retrieved successfully.

- 400: The server will respond with HTTP 400 in case of a malformed request,

Globally adjusts the AQL query result cache properties

changes the configuration for the AQL query cache

PUT /_api/query-cache/properties

After the properties have been changed, the current set of properties will be returned in the HTTP response.

Note: changing the properties may invalidate all results in the cache. The global properties for AQL query cache. The properties need to be passed in the attribute properties in the body of the HTTP request. properties needs to be a JSON object with the following properties:

Body Parameters

- mode: the mode the AQL query cache should operate in. Possible values are off, on or demand.

- maxResults: the maximum number of query results that will be stored per database-specific cache.

Return Codes

- 200: Is returned if the properties were changed successfully.

- 400: The server will respond with HTTP 400 in case of a malformed request,

HTTP Interface for AQL User Functions Management

AQL User Functions Management

This is an introduction to ArangoDB's HTTP interface for managing AQL user functions. AQL user functions are a means to extend the functionality of ArangoDB's query language (AQL) with user-defined JavaScript code.

For an overview of how AQL user functions work, please refer to Extending Aql.

The HTTP interface provides an API for adding, deleting, and listing previously registered AQL user functions.

All user functions managed through this interface will be stored in the system collection _aqlfunctions. Documents in this collection should not be accessed directly, but only via the dedicated interfaces.

Create AQL user function

create a new AQL user function

POST /_api/aqlfunction

Body Parameters

- isDeterministic: an optional boolean value to indicate that the function results are fully deterministic (function return value solely depends on the input value and return value is the same for repeated calls with same input). The isDeterministic attribute is currently not used but may be used later for optimisations.

- code: a string representation of the function body.

- name: the fully qualified name of the user functions.

In case of success, the returned JSON object has the following properties:

- error: boolean flag to indicate that an error occurred (false in this case)

- code: the HTTP status code

The body of the response will contain a JSON object with additional error details. The object has the following attributes:

- error: boolean flag to indicate that an error occurred (true in this case)

- code: the HTTP status code

- errorNum: the server error number

- errorMessage: a descriptive error message

Return Codes

- 200: If the function already existed and was replaced by the call, the server will respond with HTTP 200.

- 201: If the function can be registered by the server, the server will respond with HTTP 201.

- 400: If the JSON representation is malformed or mandatory data is missing from the request, the server will respond with HTTP 400.

Examples

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/aqlfunction <<EOF

{

"name" : "myfunctions::temperature::celsiustofahrenheit",

"code" : "function (celsius) { return celsius * 1.8 + 32; }"

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

{

"error" : false,

"code" : 201

}

Remove existing AQL user function

remove an existing AQL user function

DELETE /_api/aqlfunction/{name}

Path Parameters

- name (required): the name of the AQL user function.

Query Parameters

- group (optional): If set to true, then the function name provided in name is treated as a namespace prefix, and all functions in the specified namespace will be deleted. If set to false, the function name provided in name must be fully qualified, including any namespaces.

Removes an existing AQL user function, identified by name.

In case of success, the returned JSON object has the following properties:

- error: boolean flag to indicate that an error occurred (false in this case)

- code: the HTTP status code

The body of the response will contain a JSON object with additional error details. The object has the following attributes:

- error: boolean flag to indicate that an error occurred (true in this case)

- code: the HTTP status code

- errorNum: the server error number

- errorMessage: a descriptive error message

Return Codes

- 200: If the function can be removed by the server, the server will respond with HTTP 200.

- 400: If the user function name is malformed, the server will respond with HTTP 400.

- 404: If the specified user user function does not exist, the server will respond with HTTP 404.

Examples

deletes a function:

shell> curl -X DELETE --dump - http://localhost:8529/_api/aqlfunction/square::x::y

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

{

"error" : false,

"code" : 200

}

function not found:

shell> curl -X DELETE --dump - http://localhost:8529/_api/aqlfunction/myfunction::x::y

HTTP/1.1 404 Not Found

content-type: application/json; charset=utf-8

show response body

Return registered AQL user functions

gets all reqistered AQL user functions

GET /_api/aqlfunction

Query Parameters

- namespace (optional): Returns all registered AQL user functions from namespace namespace.

Returns all registered AQL user functions.

The call will return a JSON array with all user functions found. Each user function will at least have the following attributes:

- name: The fully qualified name of the user function

- code: A string representation of the function body

Return Codes

- 200: if success HTTP 200 is returned.

Examples

shell> curl --dump - http://localhost:8529/_api/aqlfunction

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

[

{

"name" : "myfunctions::temperature::celsiustofahrenheit",

"code" : "function (celsius) { return celsius * 1.8 + 32; }"

}

]

HTTP Interface for Simple Queries

Simple Queries

This is an introduction to ArangoDB's HTTP interface for simple queries.

Simple queries can be used if the query condition is straight forward simple, i.e., a document reference, all documents, a query-by-example, or a simple geo query. In a simple query you can specify exactly one collection and one condition. The result can then be sorted and can be split into pages.

Working with Simples Queries using HTTP

To limit the amount of results to be transferred in one batch, simple queries support a batchSize parameter that can optionally be used to tell the server to limit the number of results to be transferred in one batch to a certain value. If the query has more results than were transferred in one go, more results are waiting on the server so they can be fetched subsequently. If no value for the batchSize parameter is specified, the server will use a reasonable default value.

If the server has more documents than should be returned in a single batch, the server will set the hasMore attribute in the result. It will also return the id of the server-side cursor in the id attribute in the result. This id can be used with the cursor API to fetch any outstanding results from the server and dispose the server-side cursor afterwards.

Return all documents

returns all documents of a collection

PUT /_api/simple/all

Post Body query (required): Contains the query.

Returns all documents of a collections. The call expects a JSON object as body with the following attributes:

- collection: The name of the collection to query.

- skip: The number of documents to skip in the query (optional).

- limit: The maximal amount of documents to return. The skip is applied before the limit restriction. (optional)

Returns a cursor containing the result, see Http Cursor for details.

Return Codes

- 201: is returned if the query was executed successfully.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

Limit the amount of documents using limit

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/all <<EOF

{ "collection": "products", "skip": 2, "limit" : 2 }

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Using a batchSize value

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/all <<EOF

{ "collection": "products", "batchSize" : 3 }

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Simple query by-example

returns all documents of a collection matching a given example

PUT /_api/simple/by-example

Body Parameters

- skip: The number of documents to skip in the query (optional).

- limit: The maximal amount of documents to return. The skip is applied before the limit restriction. (optional)

- example: The example document.

- collection: The name of the collection to query.

This will find all documents matching a given example.

Returns a cursor containing the result, see Http Cursor for details.

Return Codes

- 201: is returned if the query was executed successfully.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

Matching an attribute

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/by-example <<EOF

{

"collection" : "products",

"example" : {

"i" : 1

}

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Matching an attribute which is a sub-document

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/by-example <<EOF

{

"collection" : "products",

"example" : {

"a.j" : 1

}

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Matching an attribute within a sub-document

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/by-example <<EOF

{

"collection" : "products",

"example" : {

"a" : {

"j" : 1

}

}

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Find documents matching an example

returns one document of a collection matching a given example

PUT /_api/simple/first-example

Body Parameters

- example: The example document.

- collection: The name of the collection to query.

This will return the first document matching a given example.

Returns a result containing the document or HTTP 404 if no document matched the example.

If more than one document in the collection matches the specified example, only one of these documents will be returned, and it is undefined which of the matching documents is returned.

Return Codes

- 200: is returned when the query was successfully executed.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

If a matching document was found

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/first-example <<EOF

{

"collection" : "products",

"example" : {

"i" : 1

}

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

If no document was found

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/first-example <<EOF

{

"collection" : "products",

"example" : {

"l" : 1

}

}

EOF

HTTP/1.1 404 Not Found

content-type: application/json; charset=utf-8

show response body

Find documents by their keys

fetches multiple documents by their keys

PUT /_api/simple/lookup-by-keys

Body Parameters

- keys: array with the _keys of documents to remove. of type string

- collection: The name of the collection to look in for the documents

Looks up the documents in the specified collection using the array of keys provided. All documents for which a matching key was specified in the keys array and that exist in the collection will be returned. Keys for which no document can be found in the underlying collection are ignored, and no exception will be thrown for them.

The body of the response contains a JSON object with a documents attribute. The documents attribute is an array containing the matching documents. The order in which matching documents are present in the result array is unspecified.

Return Codes

- 200: is returned if the operation was carried out successfully.

- 404: is returned if the collection was not found. The response body contains an error document in this case.

- 405: is returned if the operation was called with a different HTTP METHOD than PUT.

Examples

Looking up existing documents

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/lookup-by-keys <<EOF

{

"keys" : [

"test0",

"test1",

"test2",

"test3",

"test4",

"test5",

"test6",

"test7",

"test8",

"test9"

],

"collection" : "test"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Looking up non-existing documents

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/lookup-by-keys <<EOF

{

"keys" : [

"foo",

"bar",

"baz"

],

"collection" : "test"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Return a random document

returns a random document from a collection

PUT /_api/simple/any

Returns a random document from a collection. The call expects a JSON object as body with the following attributes:

Body Parameters

- collection: The identifier or name of the collection to query. Returns a JSON object with the document stored in the attribute document if the collection contains at least one document. If the collection is empty, the document attrbute contains null.

Returns a JSON object with the document stored in the attribute document if the collection contains at least one document. If the collection is empty, the document attrbute contains null.

Return Codes

- 200: is returned if the query was executed successfully.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/any <<EOF

{

"collection" : "products"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Remove documents by their keys

removes multiple documents by their keys

PUT /_api/simple/remove-by-keys

Body Parameters

- keys: array with the _keys of documents to remove. of type string

- options:

- waitForSync: if set to true, then all removal operations will instantly be synchronized to disk. If this is not specified, then the collection's default sync behavior will be applied.

- collection: The name of the collection to look in for the documents to remove

Looks up the documents in the specified collection using the array of keys provided, and removes all documents from the collection whose keys are contained in the keys array. Keys for which no document can be found in the underlying collection are ignored, and no exception will be thrown for them.

The body of the response contains a JSON object with information how many documents were removed (and how many were not). The removed attribute will contain the number of actually removed documents. The ignored attribute will contain the number of keys in the request for which no matching document could be found.

Return Codes

- 200: is returned if the operation was carried out successfully. The number of removed documents may still be 0 in this case if none of the specified document keys were found in the collection.

- 404: is returned if the collection was not found. The response body contains an error document in this case.

- 405: is returned if the operation was called with a different HTTP METHOD than PUT.

Examples

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/remove-by-keys <<EOF

{

"keys" : [

"test0",

"test1",

"test2",

"test3",

"test4",

"test5",

"test6",

"test7",

"test8",

"test9"

],

"collection" : "test"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/remove-by-keys <<EOF

{

"keys" : [

"foo",

"bar",

"baz"

],

"collection" : "test"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Remove documents by example

removes all documents of a collection that match an example

PUT /_api/simple/remove-by-example

Body Parameters

- example: An example document that all collection documents are compared against.

- collection: The name of the collection to remove from.

- options:

- limit: an optional value that determines how many documents to delete at most. If limit is specified but is less than the number of documents in the collection, it is undefined which of the documents will be deleted.

- waitForSync: if set to true, then all removal operations will instantly be synchronized to disk. If this is not specified, then the collection's default sync behavior will be applied.

This will find all documents in the collection that match the specified example object.

Note: the limit attribute is not supported on sharded collections. Using it will result in an error.

Returns the number of documents that were deleted.

Return Codes

- 200: is returned if the query was executed successfully.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/remove-by-example <<EOF

{

"collection" : "products",

"example" : {

"a" : {

"j" : 1

}

}

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Using Parameter: waitForSync and limit

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/remove-by-example <<EOF

{

"collection" : "products",

"example" : {

"a" : {

"j" : 1

}

},

"waitForSync" : true,

"limit" : 2

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Using Parameter: waitForSync and limit with new signature

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/remove-by-example <<EOF

{

"collection" : "products",

"example" : {

"a" : {

"j" : 1

}

},

"options" : {

"waitForSync" : true,

"limit" : 2

}

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Replace documents by example

replaces the body of all documents of a collection that match an example

PUT /_api/simple/replace-by-example

Body Parameters

- options:

- limit: an optional value that determines how many documents to replace at most. If limit is specified but is less than the number of documents in the collection, it is undefined which of the documents will be replaced.

- waitForSync: if set to true, then all removal operations will instantly be synchronized to disk. If this is not specified, then the collection's default sync behavior will be applied.

- example: An example document that all collection documents are compared against.

- collection: The name of the collection to replace within.

- newValue: The replacement document that will get inserted in place of the "old" documents.

This will find all documents in the collection that match the specified example object, and replace the entire document body with the new value specified. Note that document meta-attributes such as _id, _key, _from, _to etc. cannot be replaced.

Note: the limit attribute is not supported on sharded collections. Using it will result in an error.

Returns the number of documents that were replaced.

Return Codes

- 200: is returned if the query was executed successfully.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/replace-by-example <<EOF

{

"collection" : "products",

"example" : {

"a" : {

"j" : 1

}

},

"newValue" : {

"foo" : "bar"

},

"limit" : 3

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Using new Signature for attributes WaitForSync and limit

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/replace-by-example <<EOF

{

"collection" : "products",

"example" : {

"a" : {

"j" : 1

}

},

"newValue" : {

"foo" : "bar"

},

"options" : {

"limit" : 3,

"waitForSync" : true

}

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Update documents by example

partially updates the body of all documents of a collection that match an example

PUT /_api/simple/update-by-example

Body Parameters

- options:

- keepNull: This parameter can be used to modify the behavior when handling null values. Normally, null values are stored in the database. By setting the keepNull parameter to false, this behavior can be changed so that all attributes in data with null values will be removed from the updated document.

- limit: an optional value that determines how many documents to update at most. If limit is specified but is less than the number of documents in the collection, it is undefined which of the documents will be updated.

- waitForSync: if set to true, then all removal operations will instantly be synchronized to disk. If this is not specified, then the collection's default sync behavior will be applied.

- mergeObjectsc: Controls whether objects (not arrays) will be merged if present in both the existing and the patch document. If set to false, the value in the patch document will overwrite the existing document's value. If set to true, objects will be merged. The default is true.

- example: An example document that all collection documents are compared against.

- collection: The name of the collection to update within.

- newValue: A document containing all the attributes to update in the found documents.

This will find all documents in the collection that match the specified example object, and partially update the document body with the new value specified. Note that document meta-attributes such as _id, _key, _from, _to etc. cannot be replaced.

Note: the limit attribute is not supported on sharded collections. Using it will result in an error.

Returns the number of documents that were updated.

Return Codes

- 200: is returned if the collection was updated successfully and waitForSync was true.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

using old syntax for options

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/update-by-example <<EOF

{

"collection" : "products",

"example" : {

"a" : {

"j" : 1

}

},

"newValue" : {

"a" : {

"j" : 22

}

},

"limit" : 3

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

using new signature for options

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/update-by-example <<EOF

{

"collection" : "products",

"example" : {

"a" : {

"j" : 1

}

},

"newValue" : {

"a" : {

"j" : 22

}

},

"options" : {

"limit" : 3,

"waitForSync" : true

}

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

First document of a collection

returns the first document(s) of a collection

PUT /_api/simple/first

Body Parameters

- count: the number of documents to return at most. Specifying count is optional. If it is not specified, it defaults to 1.

- collection: the name of the collection

This will return the first document(s) from the collection, in the order of insertion/update time. When the count argument is supplied, the result will be an array of documents, with the "oldest" document being first in the result array. If the count argument is not supplied, the result is the "oldest" document of the collection, or null if the collection is empty.

Note: this method is not supported for sharded collections with more than one shard.

Return Codes

- 200: is returned when the query was successfully executed.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

Retrieving the first n documents

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/first <<EOF

{

"collection" : "products",

"count" : 2

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Retrieving the first document

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/first <<EOF

{

"collection" : "products"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Last document of a collection

returns the last document(s) of a collection

PUT /_api/simple/last

Body Parameters

- count: the number of documents to return at most. Specifying count is optional. If it is not specified, it defaults to 1.

- collection: the name of the collection

This will return the last documents from the collection, in the order of insertion/update time. When the count argument is supplied, the result will be an array of documents, with the "latest" document being first in the result array.

If the count argument is not supplied, the result is the "latest" document of the collection, or null if the collection is empty.

Note: this method is not supported for sharded collections with more than one shard.

Return Codes

- 200: is returned when the query was successfully executed.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

Retrieving the last n documents

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/last <<EOF

{

"collection" : "products",

"count" : 2

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Retrieving the first document

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/last <<EOF

{

"collection" : "products"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Simple range query

returns all documents of a collection within a range

PUT /_api/simple/range

Body Parameters

- right: The upper bound.

- attribute: The attribute path to check.

- collection: The name of the collection to query.

- limit: The maximal amount of documents to return. The skip is applied before the limit restriction. (optional)

- closed: If true, use interval including left and right, otherwise exclude right, but include left.

- skip: The number of documents to skip in the query (optional).

- left: The lower bound.

This will find all documents within a given range. In order to execute a range query, a skip-list index on the queried attribute must be present.

Returns a cursor containing the result, see Http Cursor for details.

Note: the range simple query is deprecated as of ArangoDB 2.6. The function may be removed in future versions of ArangoDB. The preferred way for retrieving documents from a collection within a specific range is to use an AQL query as follows:

FOR doc IN @@collection

FILTER doc.value >= @left && doc.value < @right

LIMIT @skip, @limit

RETURN doc`

Return Codes

- 201: is returned if the query was executed successfully.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown or no suitable index for the range query is present. The response body contains an error document in this case.

Examples

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/range <<EOF

{

"collection" : "products",

"attribute" : "i",

"left" : 2,

"right" : 4

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Returns documents near a coordinate

returns all documents of a collection near a given location

PUT /_api/simple/near

Body Parameters

- distance: If given, the attribute key used to return the distance to the given coordinate. (optional). If specified, distances are returned in meters.

- skip: The number of documents to skip in the query. (optional)

- longitude: The longitude of the coordinate.

- limit: The maximal amount of documents to return. The skip is applied before the limit restriction. The default is 100. (optional)

- collection: The name of the collection to query.

- latitude: The latitude of the coordinate.

- geo: If given, the identifier of the geo-index to use. (optional)

The default will find at most 100 documents near the given coordinate. The returned array is sorted according to the distance, with the nearest document being first in the return array. If there are near documents of equal distance, documents are chosen randomly from this set until the limit is reached.

In order to use the near operator, a geo index must be defined for the collection. This index also defines which attribute holds the coordinates for the document. If you have more than one geo-spatial index, you can use the geo field to select a particular index.

Returns a cursor containing the result, see Http Cursor for details.

Note: the near simple query is deprecated as of ArangoDB 2.6. This API may be removed in future versions of ArangoDB. The preferred way for retrieving documents from a collection using the near operator is to issue an AQL query using the NEAR function as follows:

FOR doc IN NEAR(@@collection, @latitude, @longitude, @limit)

RETURN doc`

Return Codes

- 201: is returned if the query was executed successfully.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

Without distance

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/near <<EOF

{

"collection" : "products",

"latitude" : 0,

"longitude" : 0,

"skip" : 1,

"limit" : 2

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

With distance

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/near <<EOF

{

"collection" : "products",

"latitude" : 0,

"longitude" : 0,

"skip" : 1,

"limit" : 3,

"distance" : "distance"

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Find documents within a radius around a coordinate

returns all documents of a collection within a given radius

PUT /_api/simple/within

Body Parameters

- distance: If given, the attribute key used to return the distance to the given coordinate. (optional). If specified, distances are returned in meters.

- skip: The number of documents to skip in the query. (optional)

- longitude: The longitude of the coordinate.

- limit: The maximal amount of documents to return. The skip is applied before the limit restriction. The default is 100. (optional)

- collection: The name of the collection to query.

- latitude: The latitude of the coordinate.

- radius: The maximal radius (in meters).

- geo: If given, the identifier of the geo-index to use. (optional)

This will find all documents within a given radius around the coordinate (latitude, longitude). The returned list is sorted by distance.

In order to use the within operator, a geo index must be defined for the collection. This index also defines which attribute holds the coordinates for the document. If you have more than one geo-spatial index, you can use the geo field to select a particular index.

Returns a cursor containing the result, see Http Cursor for details.

Note: the within simple query is deprecated as of ArangoDB 2.6. This API may be removed in future versions of ArangoDB. The preferred way for retrieving documents from a collection using the near operator is to issue an AQL query using the WITHIN function as follows:

FOR doc IN WITHIN(@@collection, @latitude, @longitude, @radius, @distanceAttributeName)

RETURN doc

Return Codes

- 201: is returned if the query was executed successfully.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

Without distance

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/near <<EOF

{

"collection" : "products",

"latitude" : 0,

"longitude" : 0,

"skip" : 1,

"limit" : 2,

"radius" : 500

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

With distance

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/near <<EOF

{

"collection" : "products",

"latitude" : 0,

"longitude" : 0,

"skip" : 1,

"limit" : 3,

"distance" : "distance",

"radius" : 300

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Within rectangle query

returns all documents of a collection within a rectangle

PUT /_api/simple/within-rectangle

Body Parameters

- latitude1: The latitude of the first rectangle coordinate.

- skip: The number of documents to skip in the query. (optional)

- latitude2: The latitude of the second rectangle coordinate.

- longitude2: The longitude of the second rectangle coordinate.

- longitude1: The longitude of the first rectangle coordinate.

- limit: The maximal amount of documents to return. The skip is applied before the limit restriction. The default is 100. (optional)

- collection: The name of the collection to query.

- geo: If given, the identifier of the geo-index to use. (optional)

This will find all documents within the specified rectangle (determined by the given coordinates (latitude1, longitude1, latitude2, longitude2).

In order to use the within-rectangle query, a geo index must be defined for the collection. This index also defines which attribute holds the coordinates for the document. If you have more than one geo-spatial index, you can use the geo field to select a particular index.

Returns a cursor containing the result, see Http Cursor for details.

Return Codes

- 201: is returned if the query was executed successfully.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/within-rectangle <<EOF

{

"collection" : "products",

"latitude1" : 0,

"longitude1" : 0,

"latitude2" : 0.2,

"longitude2" : 0.2,

"skip" : 1,

"limit" : 2

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Fulltext index query

returns documents of a collection as a result of a fulltext query

PUT /_api/simple/fulltext

Body Parameters

- index: The identifier of the fulltext-index to use.

- attribute: The attribute that contains the texts.

- collection: The name of the collection to query.

- limit: The maximal amount of documents to return. The skip is applied before the limit restriction. (optional)

- skip: The number of documents to skip in the query (optional).

- query: The fulltext query. Please refer to Fulltext queries for details.

This will find all documents from the collection that match the fulltext query specified in query.

In order to use the fulltext operator, a fulltext index must be defined for the collection and the specified attribute.

Returns a cursor containing the result, see Http Cursor for details.

Note: the fulltext simple query is deprecated as of ArangoDB 2.6. This API may be removed in future versions of ArangoDB. The preferred way for retrieving documents from a collection using the near operator is to issue an AQL query using the FULLTEXT AQL function as follows:

FOR doc IN FULLTEXT(@@collection, @attributeName, @queryString, @limit)

RETURN doc

Return Codes

- 201: is returned if the query was executed successfully.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/fulltext <<EOF

{

"collection" : "products",

"attribute" : "text",

"query" : "word"

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

HTTP Interface for Collections

Collections

This is an introduction to ArangoDB's HTTP interface for collections.

Collection

A collection consists of documents. It is uniquely identified by its collection identifier. It also has a unique name that clients should use to identify and access it. Collections can be renamed. This will change the collection name, but not the collection identifier. Collections have a type that is specified by the user when the collection is created. There are currently two types: document and edge. The default type is document.

Collection Identifier

A collection identifier lets you refer to a collection in a database. It is a string value and is unique within the database. Up to including ArangoDB 1.1, the collection identifier has been a client's primary means to access collections. Starting with ArangoDB 1.2, clients should instead use a collection's unique name to access a collection instead of its identifier. ArangoDB currently uses 64bit unsigned integer values to maintain collection ids internally. When returning collection ids to clients, ArangoDB will put them into a string to ensure the collection id is not clipped by clients that do not support big integers. Clients should treat the collection ids returned by ArangoDB as opaque strings when they store or use it locally.

Note: collection ids have been returned as integers up to including ArangoDB 1.1

Collection Name

A collection name identifies a collection in a database. It is a string and is unique within the database. Unlike the collection identifier it is supplied by the creator of the collection. The collection name must consist of letters, digits, and the _ (underscore) and - (dash) characters only. Please refer to Naming Conventions in ArangoDB for more information on valid collection names.

Key Generator

ArangoDB allows using key generators for each collection. Key generators have the purpose of auto-generating values for the _key attribute of a document if none was specified by the user. By default, ArangoDB will use the traditional key generator. The traditional key generator will auto-generate key values that are strings with ever-increasing numbers. The increment values it uses are non-deterministic.

Contrary, the auto increment key generator will auto-generate deterministic key values. Both the start value and the increment value can be defined when the collection is created. The default start value is 0 and the default increment is 1, meaning the key values it will create by default are:

1, 2, 3, 4, 5, ...

When creating a collection with the auto increment key generator and an increment of 5, the generated keys would be:

1, 6, 11, 16, 21, ...

The basic operations (create, read, update, delete) for documents are mapped to the standard HTTP methods (POST, GET, PUT, DELETE).

Address of a Collection

All collections in ArangoDB have an unique identifier and a unique name. ArangoDB internally uses the collection's unique identifier to look up collections. This identifier however is managed by ArangoDB and the user has no control over it. In order to allow users use their own names, each collection also has a unique name, which is specified by the user. To access a collection from the user perspective, the collection name should be used, i.e.:

http://server:port/_api/collection/collection-name

For example: Assume that the collection identifier is 7254820 and the collection name is demo, then the URL of that collection is:

http://localhost:8529/_api/collection/demo

Creating and Deleting Collections

Create collection

creates a collection

POST /_api/collection

Body Parameters

- journalSize: The maximal size of a journal or datafile in bytes. The value must be at least  1048576  (1 MiB). (The default is a configuration parameter)

- keyOptions:

- allowUserKeys: if set to true, then it is allowed to supply own key values in the _key attribute of a document. If set to false, then the key generator will solely be responsible for generating keys and supplying own key values in the _key attribute of documents is considered an error.

- type: specifies the type of the key generator. The currently available generators are traditional and autoincrement.

- increment: increment value for autoincrement key generator. Not used for other key generator types.

- offset: Initial offset value for autoincrement key generator. Not used for other key generator types.

- name: The name of the collection.

- waitForSync: If true then the data is synchronized to disk before returning from a document create, update, replace or removal operation. (default: false)

- doCompact: whether or not the collection will be compacted (default is true)

- isVolatile: If true then the collection data is kept in-memory only and not made persistent. Unloading the collection will cause the collection data to be discarded. Stopping or re-starting the server will also cause full loss of data in the collection. Setting this option will make the resulting collection be slightly faster than regular collections because ArangoDB does not enforce any synchronization to disk and does not calculate any CRC checksums for datafiles (as there are no datafiles). This option should therefore be used for cache-type collections only, and not for data that cannot be re-created otherwise. (The default is false)

- shardKeys: (The default is [ "_key" ]): in a cluster, this attribute determines which document attributes are used to determine the target shard for documents. Documents are sent to shards based on the values of their shard key attributes. The values of all shard key attributes in a document are hashed, and the hash value is used to determine the target shard. Note: Values of shard key attributes cannot be changed once set. This option is meaningless in a single server setup.

- numberOfShards: (The default is 1): in a cluster, this value determines the number of shards to create for the collection. In a single server setup, this option is meaningless.

- isSystem: If true, create a system collection. In this case collection-name should start with an underscore. End users should normally create non-system collections only. API implementors may be required to create system collections in very special occasions, but normally a regular collection will do. (The default is false)

- type: (The default is 2): the type of the collection to create. The following values for type are valid:

- 2: document collection

- 3: edges collection

- indexBuckets: The: number of buckets into which indexes using a hash table are split. The default is 16 and this number has to be a power of 2 and less than or equal to 1024. For very large collections one should increase this to avoid long pauses when the hash table has to be initially built or resized, since buckets are resized individually and can be initially built in parallel. For example, 64 might be a sensible value for a collection with 100 000 000 documents. Currently, only the edge index respects this value, but other index types might follow in future ArangoDB versions. Changes (see below) are applied when the collection is loaded the next time.

For very large collections one should increase this to avoid long pauses when the hash table has to be initially built or resized, since buckets are resized individually and can be initially built in parallel. For example, 64 might be a sensible value for a collection with 100 000 000 documents. Currently, only the edge index respects this value, but other index types might follow in future ArangoDB versions. Changes (see below) are applied when the collection is loaded the next time.

Creates an new collection with a given name. The request must contain an object with the following attributes.

- 400: If the collection-name is missing, then a HTTP 400 is returned.

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Examples

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/collection <<EOF

{

"name" : "testCollectionBasics"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

location: /_db/_system/_api/collection/testCollectionBasics

show response body

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/collection <<EOF

{

"name" : "testCollectionUsers",

"keyOptions" : {

"type" : "autoincrement",

"increment" : 5,

"allowUserKeys" : true

}

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

location: /_db/_system/_api/collection/testCollectionUsers

show response body

Drops collection

drops a collection

DELETE /_api/collection/{collection-name}

Path Parameters

- collection-name (required): The name of the collection to drop.

Drops the collection identified by collection-name.

If the collection was successfully dropped, an object is returned with the following attributes:

- error: false

- id: The identifier of the dropped collection.

Return Codes

- 400: If the collection-name is missing, then a HTTP 400 is returned.

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Examples

Using an identifier:

shell> curl -X DELETE --dump - http://localhost:8529/_api/collection/644280552

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Using a name:

shell> curl -X DELETE --dump - http://localhost:8529/_api/collection/products1

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Truncate collection

truncates a collection

PUT /_api/collection/{collection-name}/truncate

Path Parameters

- collection-name (required): The name of the collection.

Removes all documents from the collection, but leaves the indexes intact.

Return Codes

- 400: If the collection-name is missing, then a HTTP 400 is returned.

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Examples

shell> curl -X PUT --dump - http://localhost:8529/_api/collection/products/truncate

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Getting Information about a Collection

Return information about a collection

returns a collection

GET /_api/collection/{collection-name}

Path Parameters

- collection-name (required): The name of the collection.

The result is an object describing the collection with the following attributes:

- id: The identifier of the collection.

- name: The name of the collection.

- status: The status of the collection as number.

- 1: new born collection

- 2: unloaded

- 3: loaded

- 4: in the process of being unloaded

- 5: deleted

- 6: loading

Every other status indicates a corrupted collection.

- type: The type of the collection as number.

- 2: document collection (normal case)

- 3: edges collection

- isSystem: If true then the collection is a system collection.

Return Codes

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Read properties of a collection

reads the properties of the specified collection

GET /_api/collection/{collection-name}/properties

Path Parameters

- collection-name (required): The name of the collection.

In addition to the above, the result will always contain the waitForSync, doCompact, journalSize, and isVolatile attributes. This is achieved by forcing a load of the underlying collection.

- waitForSync: If true then creating, changing or removing documents will wait until the data has been synchronized to disk.

- doCompact: Whether or not the collection will be compacted.

- journalSize: The maximal size setting for journals / datafiles in bytes.

- keyOptions: JSON object which contains key generation options:

- type: specifies the type of the key generator. The currently available generators are traditional and autoincrement.

- allowUserKeys: if set to true, then it is allowed to supply own key values in the _key attribute of a document. If set to false, then the key generator is solely responsible for generating keys and supplying own key values in the _key attribute of documents is considered an error.

- isVolatile: If true then the collection data will be kept in memory only and ArangoDB will not write or sync the data to disk.

In a cluster setup, the result will also contain the following attributes:

- numberOfShards: the number of shards of the collection.

- shardKeys: contains the names of document attributes that are used to determine the target shard for documents.

Return Codes

- 400: If the collection-name is missing, then a HTTP 400 is returned.

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Examples

Using an identifier:

shell> curl --dump - http://localhost:8529/_api/collection/666824936/properties

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

location: /_db/_system/_api/collection/products/properties

show response body

Using a name:

shell> curl --dump - http://localhost:8529/_api/collection/products/properties

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

location: /_db/_system/_api/collection/products/properties

show response body

Return number of documents in a collection

Counts the documents in a collection

GET /_api/collection/{collection-name}/count

Path Parameters

- collection-name (required): The name of the collection.

In addition to the above, the result also contains the number of documents. Note that this will always load the collection into memory.

- count: The number of documents inside the collection.

Return Codes

- 400: If the collection-name is missing, then a HTTP 400 is returned.

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Examples

Requesting the number of documents:

shell> curl --dump - http://localhost:8529/_api/collection/products/count

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

location: /_db/_system/_api/collection/products/count

show response body

Return statistics for a collection

Fetch the statistics of a collection

GET /_api/collection/{collection-name}/figures

Path Parameters

- collection-name (required): The name of the collection.

In addition to the above, the result also contains the number of documents and additional statistical information about the collection. Note : This will always load the collection into memory.

Note: collection data that are stored in the write-ahead log only are not reported in the results. When the write-ahead log is collected, documents might be added to journals and datafiles of the collection, which may modify the figures of the collection.

Additionally, the filesizes of collection and index parameter JSON files are not reported. These files should normally have a size of a few bytes each. Please also note that the fileSize values are reported in bytes and reflect the logical file sizes. Some filesystems may use optimisations (e.g. sparse files) so that the actual physical file size is somewhat different. Directories and sub-directories may also require space in the file system, but this space is not reported in the fileSize results.

That means that the figures reported do not reflect the actual disk usage of the collection with 100% accuracy. The actual disk usage of a collection is normally slightly higher than the sum of the reported fileSize values. Still the sum of the fileSize values can still be used as a lower bound approximation of the disk usage.

Return Codes

- 200: Returns information about the collection:

Reply Body

- count: The number of documents currently present in the collection.

- journalSize: The maximal size of a journal or datafile in bytes.

- figures:

- datafiles:

- count: The number of datafiles.

- fileSize: The total filesize of datafiles (in bytes).

- uncollectedLogfileEntries: The number of markers in the write-ahead log for this collection that have not been transferred to journals or datafiles.

- compactionStatus:

- message: The action that was performed when the compaction was last run for the collection. This information can be used for debugging compaction issues.

- time: The point in time the compaction for the collection was last executed. This information can be used for debugging compaction issues.

- dead:

- count: The number of dead documents. This includes document versions that have been deleted or replaced by a newer version. Documents deleted or replaced that are contained the write-ahead log only are not reported in this figure.

- deletion: The total number of deletion markers. Deletion markers only contained in the write-ahead log are not reporting in this figure.

- size: The total size in bytes used by all dead documents.

- compactors:

- count: The number of compactor files.

- fileSize: The total filesize of all compactor files (in bytes).

- shapes:

- count: The total number of shapes used in the collection. This includes shapes that are not in use anymore. Shapes that are contained in the write-ahead log only are not reported in this figure.

- size: The total size of all shapes (in bytes). This includes shapes that are not in use anymore. Shapes that are contained in the write-ahead log only are not reported in this figure.

- waitingFor: An optional string value that contains information about which object type is at the head of the collection's cleanup queue. This information can be used for debugging compaction and unload issues.

- alive:

- count: The number of currently active documents in all datafiles and journals of the collection. Documents that are contained in the write-ahead log only are not reported in this figure.

- size: The total size in bytes used by all active documents of the collection. Documents that are contained in the write-ahead log only are not reported in this figure.

- documentReferences: The number of references to documents in datafiles that JavaScript code currently holds. This information can be used for debugging compaction and unload issues.

- attributes:

- count: The total number of attributes used in the collection. Note: the value includes data of attributes that are not in use anymore. Attributes that are contained in the write-ahead log only are not reported in this figure.

- size: The total size of the attribute data (in bytes). Note: the value includes data of attributes that are not in use anymore. Attributes that are contained in the write-ahead log only are not reported in this figure.

- shapefiles:

- count: The number of shape files. This value is deprecated and kept for compatibility reasons only. The value will always be 0 since ArangoDB 2.0 and higher.

- fileSize: The total filesize of the shape files. This value is deprecated and kept for compatibility reasons only. The value will always be 0 in ArangoDB 2.0 and higher.

- indexes:

- count: The total number of indexes defined for the collection, including the pre-defined indexes (e.g. primary index).

- size: The total memory allocated for indexes in bytes.

- journals:

- count: The number of journal files.

- fileSize: The total filesize of all journal files (in bytes).

- maxTick: The tick of the last marker that was stored in a journal of the collection. This might be 0 if the collection does not yet have a journal.

- 400: If the collection-name is missing, then a HTTP 400 is returned.

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Examples

Using an identifier and requesting the figures of the collection:

shell> curl --dump - http://localhost:8529/_api/collection/products/figures

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

location: /_db/_system/_api/collection/products/figures

show response body

Return collection revision id

Retrieve the collections revision id

GET /_api/collection/{collection-name}/revision

Path Parameters

- collection-name (required): The name of the collection.

In addition to the above, the result will also contain the collection's revision id. The revision id is a server-generated string that clients can use to check whether data in a collection has changed since the last revision check.

- revision: The collection revision id as a string.

Return Codes

- 400: If the collection-name is missing, then a HTTP 400 is returned.

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Examples

Retrieving the revision of a collection

shell> curl --dump - http://localhost:8529/_api/collection/products/revision

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Return checksum for the collection

returns a checksum for the specified collection

GET /_api/collection/{collection-name}/checksum

Path Parameters

- collection-name (required): The name of the collection.

Query Parameters

- withRevisions (optional): Whether or not to include document revision ids in the checksum calculation.

- withData (optional): Whether or not to include document body data in the checksum calculation.

Will calculate a checksum of the meta-data (keys and optionally revision ids) and optionally the document data in the collection.

The checksum can be used to compare if two collections on different ArangoDB instances contain the same contents. The current revision of the collection is returned too so one can make sure the checksums are calculated for the same state of data.

By default, the checksum will only be calculated on the _key system attribute of the documents contained in the collection. For edge collections, the system attributes _from and _to will also be included in the calculation.

By setting the optional query parameter withRevisions to true, then revision ids (_rev system attributes) are included in the checksumming.

By providing the optional query parameter withData with a value of true, the user-defined document attributes will be included in the calculation too. Note: Including user-defined attributes will make the checksumming slower.

The response is a JSON object with the following attributes:

- checksum: The calculated checksum as a number.

- revision: The collection revision id as a string.

Note: this method is not available in a cluster.

Return Codes

- 400: If the collection-name is missing, then a HTTP 400 is returned.

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Examples

Retrieving the checksum of a collection:

shell> curl --dump - http://localhost:8529/_api/collection/products/checksum

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Retrieving the checksum of a collection including the collection data, but not the revisions:

shell> curl --dump - http://localhost:8529/_api/collection/products/checksum?withRevisions=false&withData=true

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

reads all collections

returns all collections

GET /_api/collection

Query Parameters

- excludeSystem (optional): Whether or not system collections should be excluded from the result.

Returns an object with an attribute collections containing an array of all collection descriptions. The same information is also available in the names as an object with the collection names as keys.

By providing the optional query parameter excludeSystem with a value of true, all system collections will be excluded from the response.

Return Codes

- 200: The list of collections

Examples

Return information about all collections:

shell> curl --dump - http://localhost:8529/_api/collection

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Modifying a Collection

Load collection

loads a collection

PUT /_api/collection/{collection-name}/load

Path Parameters

- collection-name (required): The name of the collection.

Loads a collection into memory. Returns the collection on success.

The request body object might optionally contain the following attribute:

- count: If set, this controls whether the return value should include the number of documents in the collection. Setting count to false may speed up loading a collection. The default value for count is true.

On success an object with the following attributes is returned:

- id: The identifier of the collection.

- name: The name of the collection.

- count: The number of documents inside the collection. This is only returned if the count input parameters is set to true or has not been specified.

- status: The status of the collection as number.

- type: The collection type. Valid types are:

- 2: document collection

- 3: edges collection

- isSystem: If true then the collection is a system collection.

Return Codes

- 400: If the collection-name is missing, then a HTTP 400 is returned.

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Examples

shell> curl -X PUT --dump - http://localhost:8529/_api/collection/products/load

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Unload collection

unloads a collection

PUT /_api/collection/{collection-name}/unload

Path Parameters

- collection-name (required):

Removes a collection from memory. This call does not delete any documents. You can use the collection afterwards; in which case it will be loaded into memory, again. On success an object with the following attributes is returned:

- id: The identifier of the collection.

- name: The name of the collection.

- status: The status of the collection as number.

- type: The collection type. Valid types are:

- 2: document collection

- 3: edges collection

- isSystem: If true then the collection is a system collection.

Return Codes

- 400: If the collection-name is missing, then a HTTP 400 is returned.

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Examples

shell> curl -X PUT --dump - http://localhost:8529/_api/collection/products/unload

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Change properties of a collection

changes a collection

PUT /_api/collection/{collection-name}/properties

Path Parameters

- collection-name (required): The name of the collection.

Changes the properties of a collection. Expects an object with the attribute(s)

- waitForSync: If true then creating or changing a document will wait until the data has been synchronized to disk.

- journalSize: The maximal size of a journal or datafile in bytes. The value must be at least  1048576  (1 MB). Note that when changing the journalSize value, it will only have an effect for additional journals or datafiles that are created. Already existing journals or datafiles will not be affected.

On success an object with the following attributes is returned:

- id: The identifier of the collection.

- name: The name of the collection.

- waitForSync: The new value.

- journalSize: The new value.

- status: The status of the collection as number.

- type: The collection type. Valid types are:

- 2: document collection

- 3: edges collection

- isSystem: If true then the collection is a system collection.

- isVolatile: If true then the collection data will be kept in memory only and ArangoDB will not write or sync the data to disk.

- doCompact: Whether or not the collection will be compacted.

- keyOptions: JSON object which contains key generation options:

- type: specifies the type of the key generator. The currently available generators are traditional and autoincrement.

- allowUserKeys: if set to true, then it is allowed to supply own key values in the _key attribute of a document. If set to false, then the key generator is solely responsible for generating keys and supplying own key values in the _key attribute of documents is considered an error.

Note: some other collection properties, such as type, isVolatile, numberOfShards or shardKeys cannot be changed once a collection is created.

Return Codes

- 400: If the collection-name is missing, then a HTTP 400 is returned.

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Examples

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/collection/products/properties <<EOF

{

"waitForSync" : true

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Rename collection

renames a collection

PUT /_api/collection/{collection-name}/rename

Path Parameters

- collection-name (required): The name of the collection to rename.

Renames a collection. Expects an object with the attribute(s)

- name: The new name.

It returns an object with the attributes

- id: The identifier of the collection.

- name: The new name of the collection.

- status: The status of the collection as number.

- type: The collection type. Valid types are:

- 2: document collection

- 3: edges collection

- isSystem: If true then the collection is a system collection.

If renaming the collection succeeds, then the collection is also renamed in all graph definitions inside the  _graphs  collection in the current database.

Note: this method is not available in a cluster.

Return Codes

- 400: If the collection-name is missing, then a HTTP 400 is returned.

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Examples

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/collection/products1/rename <<EOF

{

"name" : "newname"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Rotate journal of a collection

rotates the journal of a collection

PUT /_api/collection/{collection-name}/rotate

Path Parameters

- collection-name (required): The name of the collection.

Rotates the journal of a collection. The current journal of the collection will be closed and made a read-only datafile. The purpose of the rotate method is to make the data in the file available for compaction (compaction is only performed for read-only datafiles, and not for journals).

Saving new data in the collection subsequently will create a new journal file automatically if there is no current journal.

It returns an object with the attributes

- result: will be true if rotation succeeded

Note: This method is not available in a cluster.

Return Codes

- 400: If the collection currently has no journal, HTTP 400 is returned.

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Examples

Rotating the journal:

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/collection/products/rotate <<EOF

{

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Rotating if no journal exists:

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/collection/products/rotate <<EOF

{

}

EOF

HTTP/1.1 400 Bad Request

content-type: application/json; charset=utf-8

show response body

HTTP Interface for Indexes

Indexes

This is an introduction to ArangoDB's HTTP interface for indexes in general. There are special sections for various index types.

Index

Indexes are used to allow fast access to documents. For each collection there is always the primary index which is a hash index for the document key (_key attribute). This index cannot be dropped or changed. edge collections will also have an automatically created edges index, which cannot be modified. This index provides quick access to documents via the _from and _to attributes.

Most user-land indexes can be created by defining the names of the attributes which should be indexed. Some index types allow indexing just one attribute (e.g. fulltext index) whereas other index types allow indexing multiple attributes.

Indexing system attributes such as _id, _key, _from, and _to in user-defined indexes is not supported by any index type. Manually creating an index that relies on any of these attributes is unsupported.

Index Handle

An index handle uniquely identifies an index in the database. It is a string and consists of a collection name and an index identifier separated by /. Geo Index: A geo index is used to find places on the surface of the earth fast. Hash Index: A hash index is used to find documents based on examples. A hash index can be created for one or multiple document attributes. A hash index will only be used by queries if all indexed attributes are present in the example or search query, and if all attributes are compared using the equality (== operator). That means the hash index does not support range queries.

If the index is declared unique, then access to the indexed attributes should be fast. The performance degrades if the indexed attribute(s) contain(s) only very few distinct values.

Edges Index

An edges index is automatically created for edge collections. It contains connections between vertex documents and is invoked when the connecting edges of a vertex are queried. There is no way to explicitly create or delete edge indexes.

Skiplist Index:

A skiplist is used to find ranges of documents.

Fulltext Index:

A fulltext index can be used to find words, or prefixes of words inside documents. A fulltext index can be set on one attribute only, and will index all words contained in documents that have a textual value in this attribute. Only words with a (specifiable) minimum length are indexed. Word tokenization is done using the word boundary analysis provided by libicu, which is taking into account the selected language provided at server start. Words are indexed in their lower-cased form. The index supports complete match queries (full words) and prefix queries.

The basic operations (create, read, update, delete) for documents are mapped to the standard HTTP methods (POST, GET, PUT, DELETE).

Address of an Index

All indexes in ArangoDB have an unique handle. This index handle identifies an index and is managed by ArangoDB. All indexes are found under the URI

http://server:port/_api/index/index-handle

For example: Assume that the index handle is demo/63563528 then the URL of that index is:

http://localhost:8529/_api/index/demo/63563528

Working with Indexes using HTTP

Read index

returns an index

GET /_api/index/{index-handle}

Path Parameters

- index-handle (required): The index-handle.

The result is an object describing the index. It has at least the following attributes:

- id: the identifier of the index

- type: the index type

All other attributes are type-dependent. For example, some indexes provide unique or sparse flags, whereas others don't. Some indexes also provide a selectivity estimate in the selectivityEstimate attribute of the result.

Return Codes

- 200: If the index exists, then a HTTP 200 is returned.

- 404: If the index does not exist, then a HTTP 404 is returned.

Examples

shell> curl --dump - http://localhost:8529/_api/index/products/0

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Create index

creates an index

POST /_api/index#general

Query Parameters

- collection (required): The collection name.

Post Body index-details (required):

NOTE Swagger examples won't work due to the anchor.

Creates a new index in the collection collection. Expects an object containing the index details.

The type of the index to be created must specified in the type attribute of the index details. Depending on the index type, additional other attributes may need to specified in the request in order to create the index.

Most indexes (a notable exception being the cap constraint) require the array of attributes to be indexed in the fields attribute of the index details. Depending on the index type, a single attribute or multiple attributes can be indexed.

Indexing system attributes such as _id, _key, _from, and _to is not supported for user-defined indexes. Manually creating an index using any of these attributes will fail with an error.

Some indexes can be created as unique or non-unique variants. Uniqueness can be controlled for most indexes by specifying the unique flag in the index details. Setting it to true will create a unique index. Setting it to false or omitting the unique attribute will create a non-unique index.

Note: The following index types do not support uniqueness, and using the unique attribute with these types may lead to an error:

- cap constraints

- fulltext indexes

Note: Unique indexes on non-shard keys are not supported in a cluster.

Hash and skiplist indexes can optionally be created in a sparse variant. A sparse index will be created if the sparse attribute in the index details is set to true. Sparse indexes do not index documents for which any of the index attributes is either not set or is null.

Return Codes

- 200: If the index already exists, then an HTTP 200 is returned.

- 201: If the index does not already exist and could be created, then an HTTP 201 is returned.

- 400: If an invalid index description is posted or attributes are used that the target index will not support, then an HTTP 400 is returned.

- 404: If collection is unknown, then an HTTP 404 is returned.

Delete index

deletes an index

DELETE /_api/index/{index-handle}

Path Parameters

- index-handle (required): The index handle.

Deletes an index with index-handle.

Return Codes

- 200: If the index could be deleted, then an HTTP 200 is returned.

- 404: If the index-handle is unknown, then an HTTP 404 is returned.

Examples

shell> curl -X DELETE --dump - http://localhost:8529/_api/index/products/785576168

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Read all indexes of a collection

returns all indexes of a collection

GET /_api/index

Query Parameters

- collection (required): The collection name.

Returns an object with an attribute indexes containing an array of all index descriptions for the given collection. The same information is also available in the identifiers as an object with the index handles as keys.

Return Codes

- 200: returns a json object containing a list of indexes on that collection.

Examples

Return information about all indexes

shell> curl --dump - http://localhost:8529/_api/index?collection=products

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Working with Cap Constraints

Create cap constraint

creates a cap constraint

POST /_api/index#CapConstraints

Query Parameters

- collection (required): The collection name.

Body Parameters

- byteSize: The maximal size of the active document data in the collection (in bytes). If specified, the value must be at least 16384.

- type: must be equal to "cap".

- size: The maximal number of documents for the collection. If specified, the value must be greater than zero.

NOTE Swagger examples won't work due to the anchor.

Creates a cap constraint for the collection collection-name, if it does not already exist. Expects an object containing the index details.

Note: The cap constraint does not index particular attributes of the documents in a collection, but limits the number of documents in the collection to a maximum value. The cap constraint thus does not support attribute names specified in the fields attribute nor uniqueness of any kind via the unique attribute.

It is allowed to specify either size or byteSize, or both at the same time. If both are specified, then the automatic document removal will be triggered by the first non-met constraint.

Return Codes

- 200: If the index already exists, then an HTTP 200 is returned.

- 201: If the index does not already exist and could be created, then an HTTP 201 is returned.

- 400: If either size or byteSize contain invalid values, then an HTTP 400 is returned.

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Examples

Creating a cap constraint

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/index?collection=products <<EOF

{

"type" : "cap",

"size" : 10

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Working with Hash Indexes

If a suitable hash index exists, then /_api/simple/by-example will use this index to execute a query-by-example.

Create hash index

creates a hash index

POST /_api/index#hash

Query Parameters

- collection-name (required): The collection name.

Body Parameters

- fields: an array of attribute paths. of type string

- unique: if true, then create a unique index.

- type: must be equal to "hash".

- sparse: if true, then create a sparse index.

NOTE Swagger examples won't work due to the anchor.

Creates a hash index for the collection collection-name if it does not already exist. The call expects an object containing the index details.

In a sparse index all documents will be excluded from the index that do not contain at least one of the specified index attributes (i.e. fields) or that have a value of null in any of the specified index attributes. Such documents will not be indexed, and not be taken into account for uniqueness checks if the unique flag is set.

In a non-sparse index, these documents will be indexed (for non-present indexed attributes, a value of null will be used) and will be taken into account for uniqueness checks if the unique flag is set.

Note: unique indexes on non-shard keys are not supported in a cluster.

Return Codes

- 200: If the index already exists, then a HTTP 200 is returned.

- 201: If the index does not already exist and could be created, then a HTTP 201 is returned.

- 400: If the collection already contains documents and you try to create a unique hash index in such a way that there are documents violating the uniqueness, then a HTTP 400 is returned.

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Examples

Creating an unique constraint

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/index?collection=products <<EOF

{

"type" : "hash",

"unique" : true,

"fields" : [

"a",

"b"

]

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Creating a non-unique hash index

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/index?collection=products <<EOF

{

"type" : "hash",

"unique" : false,

"fields" : [

"a",

"b"

]

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Creating a sparse index

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/index?collection=products <<EOF

{

"type" : "hash",

"unique" : false,

"sparse" : true,

"fields" : [

"a"

]

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Simple query by-example

returns all documents of a collection matching a given example

PUT /_api/simple/by-example

Body Parameters

- skip: The number of documents to skip in the query (optional).

- limit: The maximal amount of documents to return. The skip is applied before the limit restriction. (optional)

- example: The example document.

- collection: The name of the collection to query.

This will find all documents matching a given example.

Returns a cursor containing the result, see Http Cursor for details.

Return Codes

- 201: is returned if the query was executed successfully.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

Matching an attribute

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/by-example <<EOF

{

"collection" : "products",

"example" : {

"i" : 1

}

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Matching an attribute which is a sub-document

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/by-example <<EOF

{

"collection" : "products",

"example" : {

"a.j" : 1

}

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Matching an attribute within a sub-document

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/by-example <<EOF

{

"collection" : "products",

"example" : {

"a" : {

"j" : 1

}

}

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Find documents matching an example

returns one document of a collection matching a given example

PUT /_api/simple/first-example

Body Parameters

- example: The example document.

- collection: The name of the collection to query.

This will return the first document matching a given example.

Returns a result containing the document or HTTP 404 if no document matched the example.

If more than one document in the collection matches the specified example, only one of these documents will be returned, and it is undefined which of the matching documents is returned.

Return Codes

- 200: is returned when the query was successfully executed.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

If a matching document was found

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/first-example <<EOF

{

"collection" : "products",

"example" : {

"i" : 1

}

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

If no document was found

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/first-example <<EOF

{

"collection" : "products",

"example" : {

"l" : 1

}

}

EOF

HTTP/1.1 404 Not Found

content-type: application/json; charset=utf-8

show response body

Working with Skiplist Indexes

If a suitable skip-list index exists, then /_api/simple/range will use this index to execute a range query.

Create skip list

creates a skip-list

POST /_api/index

Query Parameters

- collection-name (required): The collection name.

Body Parameters

- fields: an array of attribute paths. of type string

- unique: if true, then create a unique index.

- type: must be equal to "skiplist".

- sparse: if true, then create a sparse index.

Creates a skip-list index for the collection collection-name, if it does not already exist. The call expects an object containing the index details.

In a sparse index all documents will be excluded from the index that do not contain at least one of the specified index attributes (i.e. fields) or that have a value of null in any of the specified index attributes. Such documents will not be indexed, and not be taken into account for uniqueness checks if the unique flag is set.

In a non-sparse index, these documents will be indexed (for non-present indexed attributes, a value of null will be used) and will be taken into account for uniqueness checks if the unique flag is set.

Note: unique indexes on non-shard keys are not supported in a cluster.

Return Codes

- 200: If the index already exists, then a HTTP 200 is returned.

- 201: If the index does not already exist and could be created, then a HTTP 201 is returned.

- 400: If the collection already contains documents and you try to create a unique skip-list index in such a way that there are documents violating the uniqueness, then a HTTP 400 is returned.

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Examples

Creating a skiplist index

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/index?collection=products <<EOF

{

"type" : "skiplist",

"unique" : false,

"fields" : [

"a",

"b"

]

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Creating a sparse skiplist index

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/index?collection=products <<EOF

{

"type" : "skiplist",

"unique" : false,

"sparse" : true,

"fields" : [

"a"

]

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Working with Geo Indexes

Create geo-spatial index

creates a geo index

POST /_api/index#geo

Query Parameters

- collection (required): The collection name.

Body Parameters

- fields: An array with one or two attribute paths. If it is an array with one attribute path location, then a geo-spatial index on all documents is created using location as path to the coordinates. The value of the attribute must be an array with at least two double values. The array must contain the latitude (first value) and the longitude (second value). All documents, which do not have the attribute path or with value that are not suitable, are ignored. If it is an array with two attribute paths latitude and longitude, then a geo-spatial index on all documents is created using latitude and longitude as paths the latitude and the longitude. The value of the attribute latitude and of the attribute longitude must a double. All documents, which do not have the attribute paths or which values are not suitable, are ignored. of type string

- type: must be equal to "geo".

- geoJson: If a geo-spatial index on a location is constructed and geoJson is true, then the order within the array is longitude followed by latitude. This corresponds to the format described in http://geojson.org/geojson-spec.html#positions

If it is an array with one attribute path location, then a geo-spatial index on all documents is created using location as path to the coordinates. The value of the attribute must be an array with at least two double values. The array must contain the latitude (first value) and the longitude (second value). All documents, which do not have the attribute path or with value that are not suitable, are ignored.

If it is an array with two attribute paths latitude and longitude, then a geo-spatial index on all documents is created using latitude and longitude as paths the latitude and the longitude. The value of the attribute latitude and of the attribute longitude must a double. All documents, which do not have the attribute paths or which values are not suitable, are ignored.

NOTE Swagger examples won't work due to the anchor.

Creates a geo-spatial index in the collection collection-name, if it does not already exist. Expects an object containing the index details.

Geo indexes are always sparse, meaning that documents that do not contain the index attributes or have non-numeric values in the index attributes will not be indexed.

Return Codes

- 200: If the index already exists, then a HTTP 200 is returned.

- 201: If the index does not already exist and could be created, then a HTTP 201 is returned.

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Examples

Creating a geo index with a location attribute

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/index?collection=products <<EOF

{

"type" : "geo",

"fields" : [

"b"

]

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Creating a geo index with latitude and longitude attributes

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/index?collection=products <<EOF

{

"type" : "geo",

"fields" : [

"e",

"f"

]

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Returns documents near a coordinate

returns all documents of a collection near a given location

PUT /_api/simple/near

Body Parameters

- distance: If given, the attribute key used to return the distance to the given coordinate. (optional). If specified, distances are returned in meters.

- skip: The number of documents to skip in the query. (optional)

- longitude: The longitude of the coordinate.

- limit: The maximal amount of documents to return. The skip is applied before the limit restriction. The default is 100. (optional)

- collection: The name of the collection to query.

- latitude: The latitude of the coordinate.

- geo: If given, the identifier of the geo-index to use. (optional)

The default will find at most 100 documents near the given coordinate. The returned array is sorted according to the distance, with the nearest document being first in the return array. If there are near documents of equal distance, documents are chosen randomly from this set until the limit is reached.

In order to use the near operator, a geo index must be defined for the collection. This index also defines which attribute holds the coordinates for the document. If you have more than one geo-spatial index, you can use the geo field to select a particular index.

Returns a cursor containing the result, see Http Cursor for details.

Note: the near simple query is deprecated as of ArangoDB 2.6. This API may be removed in future versions of ArangoDB. The preferred way for retrieving documents from a collection using the near operator is to issue an AQL query using the NEAR function as follows:

FOR doc IN NEAR(@@collection, @latitude, @longitude, @limit)

RETURN doc`

Return Codes

- 201: is returned if the query was executed successfully.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

Without distance

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/near <<EOF

{

"collection" : "products",

"latitude" : 0,

"longitude" : 0,

"skip" : 1,

"limit" : 2

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

With distance

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/near <<EOF

{

"collection" : "products",

"latitude" : 0,

"longitude" : 0,

"skip" : 1,

"limit" : 3,

"distance" : "distance"

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Find documents within a radius around a coordinate

returns all documents of a collection within a given radius

PUT /_api/simple/within

Body Parameters

- distance: If given, the attribute key used to return the distance to the given coordinate. (optional). If specified, distances are returned in meters.

- skip: The number of documents to skip in the query. (optional)

- longitude: The longitude of the coordinate.

- limit: The maximal amount of documents to return. The skip is applied before the limit restriction. The default is 100. (optional)

- collection: The name of the collection to query.

- latitude: The latitude of the coordinate.

- radius: The maximal radius (in meters).

- geo: If given, the identifier of the geo-index to use. (optional)

This will find all documents within a given radius around the coordinate (latitude, longitude). The returned list is sorted by distance.

In order to use the within operator, a geo index must be defined for the collection. This index also defines which attribute holds the coordinates for the document. If you have more than one geo-spatial index, you can use the geo field to select a particular index.

Returns a cursor containing the result, see Http Cursor for details.

Note: the within simple query is deprecated as of ArangoDB 2.6. This API may be removed in future versions of ArangoDB. The preferred way for retrieving documents from a collection using the near operator is to issue an AQL query using the WITHIN function as follows:

FOR doc IN WITHIN(@@collection, @latitude, @longitude, @radius, @distanceAttributeName)

RETURN doc

Return Codes

- 201: is returned if the query was executed successfully.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

Without distance

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/near <<EOF

{

"collection" : "products",

"latitude" : 0,

"longitude" : 0,

"skip" : 1,

"limit" : 2,

"radius" : 500

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

With distance

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/near <<EOF

{

"collection" : "products",

"latitude" : 0,

"longitude" : 0,

"skip" : 1,

"limit" : 3,

"distance" : "distance",

"radius" : 300

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Fulltext

If a fulltext index exists, then /_api/simple/fulltext will use this index to execute the specified fulltext query.

Create fulltext index

creates a fulltext index

POST /_api/index#fulltext

Query Parameters

- collection-name (required): The collection name.

Body Parameters

- fields: an array of attribute names. Currently, the array is limited to exactly one attribute. of type string

- type: must be equal to "fulltext".

- minLength: Minimum character length of words to index. Will default to a server-defined value if unspecified. It is thus recommended to set this value explicitly when creating the index.

NOTE Swagger examples won't work due to the anchor.

Creates a fulltext index for the collection collection-name, if it does not already exist. The call expects an object containing the index details.

Return Codes

- 200: If the index already exists, then a HTTP 200 is returned.

- 201: If the index does not already exist and could be created, then a HTTP 201 is returned.

- 404: If the collection-name is unknown, then a HTTP 404 is returned.

Examples

Creating a fulltext index

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/index?collection=products <<EOF

{

"type" : "fulltext",

"fields" : [

"text"

]

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Fulltext index query

returns documents of a collection as a result of a fulltext query

PUT /_api/simple/fulltext

Body Parameters

- index: The identifier of the fulltext-index to use.

- attribute: The attribute that contains the texts.

- collection: The name of the collection to query.

- limit: The maximal amount of documents to return. The skip is applied before the limit restriction. (optional)

- skip: The number of documents to skip in the query (optional).

- query: The fulltext query. Please refer to Fulltext queries for details.

This will find all documents from the collection that match the fulltext query specified in query.

In order to use the fulltext operator, a fulltext index must be defined for the collection and the specified attribute.

Returns a cursor containing the result, see Http Cursor for details.

Note: the fulltext simple query is deprecated as of ArangoDB 2.6. This API may be removed in future versions of ArangoDB. The preferred way for retrieving documents from a collection using the near operator is to issue an AQL query using the FULLTEXT AQL function as follows:

FOR doc IN FULLTEXT(@@collection, @attributeName, @queryString, @limit)

RETURN doc

Return Codes

- 201: is returned if the query was executed successfully.

- 400: is returned if the body does not contain a valid JSON representation of a query. The response body contains an error document in this case.

- 404: is returned if the collection specified by collection is unknown. The response body contains an error document in this case.

Examples

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/simple/fulltext <<EOF

{

"collection" : "products",

"attribute" : "text",

"query" : "word"

}

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

HTTP Interface for Transactions

Transactions

ArangoDB's transactions are executed on the server. Transactions can be initiated by clients by sending the transaction description for execution to the server.

Transactions in ArangoDB do not offer separate BEGIN, COMMIT and ROLLBACK operations as they are available in many other database products. Instead, ArangoDB transactions are described by a JavaScript function, and the code inside the JavaScript function will then be executed transactionally. At the end of the function, the transaction is automatically committed, and all changes done by the transaction will be persisted. If an exception is thrown during transaction execution, all operations performed in the transaction are rolled back.

For a more detailed description of how transactions work in ArangoDB please refer to Transactions.

Execute transaction

execute a server-side transaction

POST /_api/transaction

Body Parameters

- action: the actual transaction operations to be executed, in the form of stringified JavaScript code. The code will be executed on server side, with late binding. It is thus critical that the code specified in action properly sets up all the variables it needs. If the code specified in action ends with a return statement, the value returned will also be returned by the REST API in the result attribute if the transaction committed successfully.

- params: optional arguments passed to action.

- collections: contains the array of collections to be used in the transaction (mandatory). collections must be a JSON object that can have the optional sub-attributes read and write. read and write must each be either arrays of collections names or strings with a single collection name.

- lockTimeout: an optional numeric value that can be used to set a timeout for waiting on collection locks. If not specified, a default value will be used. Setting lockTimeout to 0 will make ArangoDB not time out waiting for a lock.

- waitForSync: an optional boolean flag that, if set, will force the transaction to write all data to disk before returning.

Contains the collections and action.

The transaction description must be passed in the body of the POST request.

If the transaction is fully executed and committed on the server, HTTP 200 will be returned. Additionally, the return value of the code defined in action will be returned in the result attribute.

For successfully committed transactions, the returned JSON object has the following properties:

- error: boolean flag to indicate if an error occurred (false in this case)

- code: the HTTP status code

- result: the return value of the transaction

If the transaction specification is either missing or malformed, the server will respond with HTTP 400.

The body of the response will then contain a JSON object with additional error details. The object has the following attributes:

- error: boolean flag to indicate that an error occurred (true in this case)

- code: the HTTP status code

- errorNum: the server error number

- errorMessage: a descriptive error message

If a transaction fails to commit, either by an exception thrown in the action code, or by an internal error, the server will respond with an error. Any other errors will be returned with any of the return codes HTTP 400, HTTP 409, or HTTP 500.

Return Codes

- 200: If the transaction is fully executed and committed on the server, HTTP 200 will be returned.

- 400: If the transaction specification is either missing or malformed, the server will respond with HTTP 400.

- 404: If the transaction specification contains an unknown collection, the server will respond with HTTP 404.

- 500: Exceptions thrown by users will make the server respond with a return code of HTTP 500

Examples

Executing a transaction on a single collection

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/transaction <<EOF

{

"collections" : {

"write" : "products"

},

"action" : "function () { var db = require('internal').db; db.products.save({});  return db.products.count(); }"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Executing a transaction using multiple collections

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/transaction <<EOF

{

"collections" : {

"write" : [

"products",

"materials"

]

},

"action" : "function () {var db = require('internal').db;db.products.save({});db.materials.save({});return 'worked!';}"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Aborting a transaction due to an internal error

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/transaction <<EOF

{

"collections" : {

"write" : "products"

},

"action" : "function () {var db = require('internal').db;db.products.save({ _key: 'abc'});db.products.save({ _key: 'abc'});}"

}

EOF

HTTP/1.1 400 Bad Request

content-type: application/json; charset=utf-8

show response body

Aborting a transaction by explicitly throwing an exception

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/transaction <<EOF

{

"collections" : {

"read" : "products"

},

"action" : "function () { throw 'doh!'; }"

}

EOF

HTTP/1.1 500 Internal Server Error

content-type: application/json; charset=utf-8

show response body

Referring to a non-existing collection

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/transaction <<EOF

{

"collections" : {

"read" : "products"

},

"action" : "function () { return true; }"

}

EOF

HTTP/1.1 404 Not Found

content-type: application/json; charset=utf-8

show response body

General Graphs

This chapter describes the REST interface for the multi-collection graph module. It allows you to define a graph that is spread across several edge and document collections. There is no need to include the referenced collections within the query, this module will handle it for you.

Manage your graphs

The graph module provides functions dealing with graph structures. Examples will explain the REST API on the social graph:

List all graphs

Lists all graphs known to the graph module.

GET /_api/gharial

Lists all graph names stored in this database.

Return Codes

- 200: Is returned if the module is available and the graphs could be listed.

Examples

shell> curl --dump - http://localhost:8529/_api/gharial

HTTP/1.1 200 OK

content-type: application/json

show response body

Create a graph

Create a new graph in the graph module.

POST /_api/gharial

The creation of a graph requires the name of the graph and a definition of its edges. See also edge definitions.

Body Parameters

- orphanCollections: An array of additional vertex collections.

- edgeDefinitions: An array of definitions for the edge

- name: Name of the graph.

Return Codes

- 201: Is returned if the graph could be created. The body contains the graph configuration that has been stored.

- 409: Returned if there is a conflict storing the graph. This can occur either if a graph with this name is already stored, or if there is one edge definition with a the same edge collection but a different signature used in any other graph.

Examples

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/gharial <<EOF

{

"name" : "myGraph",

"edgeDefinitions" : [

{

"collection" : "edges",

"from" : [

"startVertices"

],

"to" : [

"endVertices"

]

}

]

}

EOF

HTTP/1.1 201 Created

content-type: application/json

etag: 554430696

show response body

Get a graph

Get a graph from the graph module.

GET /_api/gharial/{graph-name}

Gets a graph from the collection _graphs. Returns the definition content of this graph.

Path Parameters

- graph-name (required): The name of the graph.

Return Codes

- 200: Returned if the graph could be found.

- 404: Returned if no graph with this name could be found.

Examples

shell> curl --dump - http://localhost:8529/_api/gharial/myGraph

HTTP/1.1 200 OK

content-type: application/json

etag: 575336680

show response body

Drop a graph

delete an existing graph

DELETE /_api/gharial/{graph-name}

Removes a graph from the collection _graphs.

Path Parameters

- graph-name (required): The name of the graph.

Query Parameters

- dropCollections (optional): Drop collections of this graph as well. Collections will only be dropped if they are not used in other graphs.

Return Codes

- 200: Returned if the graph could be dropped.

- 404: Returned if no graph with this name could be found.

Examples

shell> curl -X DELETE --dump - http://localhost:8529/_api/gharial/social?dropCollections=true

HTTP/1.1 200 OK

content-type: application/json

show response body

List vertex collections

Lists all vertex collections used in this graph.

GET /_api/gharial/{graph-name}/vertex

Lists all vertex collections within this graph.

Path Parameters

- graph-name (required): The name of the graph.

Return Codes

- 200: Is returned if the collections could be listed.

- 404: Returned if no graph with this name could be found.

Examples

shell> curl --dump - http://localhost:8529/_api/gharial/social/vertex

HTTP/1.1 200 OK

content-type: application/json

show response body

Add vertex collection

Add an additional vertex collection to the graph.

POST /_api/gharial/{graph-name}/vertex

Adds a vertex collection to the set of collections of the graph. If the collection does not exist, it will be created.

Path Parameters

- graph-name (required): The name of the graph.

Return Codes

- 201: Returned if the edge collection could be added successfully.

- 404: Returned if no graph with this name could be found.

Examples

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/gharial/social/vertex <<EOF

{

"collection" : "otherVertices"

}

EOF

HTTP/1.1 201 Created

content-type: application/json

etag: 550039784

show response body

Remove vertex collection

Remove a vertex collection form the graph.

DELETE /_api/gharial/{graph-name}/vertex/{collection-name}

Removes a vertex collection from the graph and optionally deletes the collection, if it is not used in any other graph.

Path Parameters

- graph-name (required): The name of the graph.

- collection-name (required): The name of the vertex collection.

Query Parameters

- dropCollection (optional): Drop the collection as well. Collection will only be dropped if it is not used in other graphs.

Return Codes

- 200: Returned if the vertex collection was removed from the graph successfully.

- 400: Returned if the vertex collection is still used in an edge definition. In this case it cannot be removed from the graph yet, it has to be removed from the edge definition first.

- 404: Returned if no graph with this name could be found.

Examples

You can remove vertex collections that are not used in any edge collection:

shell> curl -X DELETE --dump - http://localhost:8529/_api/gharial/social/vertex/otherVertices

HTTP/1.1 200 OK

content-type: application/json

etag: 612102376

show response body

You cannot remove vertex collections that are used in edge collections:

shell> curl -X DELETE --dump - http://localhost:8529/_api/gharial/social/vertex/male

HTTP/1.1 400 Bad Request

content-type: application/json

show response body

List edge definitions

Lists all edge definitions

GET /_api/gharial/{graph-name}/edge

Lists all edge collections within this graph.

Path Parameters

- graph-name (required): The name of the graph.

Return Codes

- 200: Is returned if the edge definitions could be listed.

- 404: Returned if no graph with this name could be found.

Examples

shell> curl --dump - http://localhost:8529/_api/gharial/social/edge

HTTP/1.1 200 OK

content-type: application/json

show response body

Add edge definition

Add a new edge definition to the graph

POST /_api/gharial/{graph-name}/edge

Adds an additional edge definition to the graph. This edge definition has to contain a collection and an array of each from and to vertex collections. An edge definition can only be added if this definition is either not used in any other graph, or it is used with exactly the same definition. It is not possible to store a definition "e" from "v1" to "v2" in the one graph, and "e" from "v2" to "v1" in the other graph.

Path Parameters

- graph-name (required): The name of the graph.

Body Parameters

- to: One or many edge collections that can contain target vertices. of type string

- from: One or many vertex collections that can contain source vertices. of type string

- collection: The name of the edge collection to be used.

Return Codes

- 200: Returned if the definition could be added successfully.

- 400: Returned if the defininition could not be added, the edge collection is used in an other graph with a different signature.

- 404: Returned if no graph with this name could be found.

Examples

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/gharial/social/edge <<EOF

{

"collection" : "lives_in",

"from" : [

"female",

"male"

],

"to" : [

"city"

]

}

EOF

HTTP/1.1 201 Created

content-type: application/json

etag: 541716712

show response body

Replace an edge definition

Replace an existing edge definition

POST /_api/gharial/{graph-name}/edge/{definition-name}

Change one specific edge definition. This will modify all occurrences of this definition in all graphs known to your database.

Path Parameters

- graph-name (required): The name of the graph.

- definition-name (required): The name of the edge collection used in the definition.

Body Parameters

- to: One or many edge collections that can contain target vertices. of type string

- from: One or many vertex collections that can contain source vertices. of type string

- collection: The name of the edge collection to be used.

Return Codes

- 200: Returned if the edge definition could be replaced.

- 400: Returned if no edge definition with this name is found in the graph.

- 404: Returned if no graph with this name could be found.

Examples

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/gharial/social/edge/relation <<EOF

{

"collection" : "relation",

"from" : [

"female",

"male",

"animal"

],

"to" : [

"female",

"male",

"animal"

]

}

EOF

HTTP/1.1 200 OK

content-type: application/json

etag: 617476328

show response body

Remove an edge definition from the graph

Remove an edge definition form the graph

DELETE /_api/gharial/{graph-name}/edge/{definition-name}

Remove one edge definition from the graph. This will only remove the edge collection, the vertex collections remain untouched and can still be used in your queries.

Path Parameters

- graph-name (required): The name of the graph.

- definition-name (required): The name of the edge collection used in the definition.

Query Parameters

- dropCollection (optional): Drop the collection as well. Collection will only be dropped if it is not used in other graphs.

Return Codes

- 200: Returned if the edge definition could be removed from the graph.

- 400: Returned if no edge definition with this name is found in the graph.

- 404: Returned if no graph with this name could be found.

Examples

shell> curl -X DELETE --dump - http://localhost:8529/_api/gharial/social/edge/relation

HTTP/1.1 200 OK

content-type: application/json

etag: 567865576

show response body

Handling Vertices

Examples will explain the REST API to the graph module on the social graph:

Create a vertex

create a new vertex

POST /_api/gharial/{graph-name}/vertex/{collection-name}

Adds a vertex to the given collection.

Path Parameters

- graph-name (required): The name of the graph.

- collection-name (required): The name of the vertex collection the vertex belongs to.

Query Parameters

- waitForSync (optional): Define if the request should wait until synced to disk.

Post Body storeThisObject (required): The body has to be the JSON object to be stored.

Return Codes

- 201: Returned if the vertex could be added and waitForSync is true.

- 202: Returned if the request was successful but waitForSync is false.

- 404: Returned if no graph or no vertex collection with this name could be found.

Examples

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/gharial/social/vertex/male <<EOF

{

"name" : "Francis"

}

EOF

HTTP/1.1 202 Accepted

content-type: application/json

etag: 548532456

show response body

Get a vertex

fetches an existing vertex

GET /_api/gharial/{graph-name}/vertex/{collection-name}/{vertex-key}

Gets a vertex from the given collection.

Path Parameters

- graph-name (required): The name of the graph.

- collection-name (required): The name of the vertex collection the vertex belongs to.

- vertex-key (required): The _key attribute of the vertex.

Header Parameters

- if-match (optional): If the "If-Match" header is given, then it must contain exactly one etag. The document is returned, if it has the same revision as the given etag. Otherwise a HTTP 412 is returned. As an alternative you can supply the etag in an attribute rev in the URL.

Return Codes

- 200: Returned if the vertex could be found.

- 404: Returned if no graph with this name, no vertex collection or no vertex with this id could be found.

- 412: Returned if if-match header is given, but the documents revision is different.

Examples

shell> curl --dump - http://localhost:8529/_api/gharial/social/vertex/female/alice

HTTP/1.1 200 OK

content-type: application/json

etag: 577171688

show response body

Modify a vertex

replace an existing vertex

PATCH /_api/gharial/{graph-name}/vertex/{collection-name}/{vertex-key}

Updates the data of the specific vertex in the collection.

Path Parameters

- graph-name (required): The name of the graph.

- collection-name (required): The name of the vertex collection the vertex belongs to.

- vertex-key (required): The _key attribute of the vertex.

Query Parameters

- waitForSync (optional): Define if the request should wait until synced to disk.

- keepNull (optional): Define if values set to null should be stored. By default the key is removed from the document.

Header Parameters

- if-match (optional): If the "If-Match" header is given, then it must contain exactly one etag. The document is updated, if it has the same revision as the given etag. Otherwise a HTTP 412 is returned. As an alternative you can supply the etag in an attribute rev in the URL.

Post Body replaceAttributes (required): The body has to contain a JSON object containing exactly the attributes that should be replaced.

Return Codes

- 200: Returned if the vertex could be updated.

- 202: Returned if the request was successful but waitForSync is false.

- 404: Returned if no graph with this name, no vertex collection or no vertex with this id could be found.

- 412: Returned if if-match header is given, but the documents revision is different.

Examples

shell> curl -X PATCH --data-binary @- --dump - http://localhost:8529/_api/gharial/social/vertex/female/alice <<EOF

{

"age" : 26

}

EOF

HTTP/1.1 202 Accepted

content-type: application/json

etag: 600240360

show response body

Replace a vertex

replaces an existing vertex

PUT /_api/gharial/{graph-name}/vertex/{collection-name}/{vertex-key}

Replaces the data of a vertex in the collection.

Path Parameters

- graph-name (required): The name of the graph.

- collection-name (required): The name of the vertex collection the vertex belongs to.

- vertex-key (required): The _key attribute of the vertex.

Query Parameters

- waitForSync (optional): Define if the request should wait until synced to disk.

Header Parameters

- if-match (optional): If the "If-Match" header is given, then it must contain exactly one etag. The document is updated, if it has the same revision as the given etag. Otherwise a HTTP 412 is returned. As an alternative you can supply the etag in an attribute rev in the URL.

Post Body storeThisJsonObject (required): The body has to be the JSON object to be stored.

Return Codes

- 200: Returned if the vertex could be replaced.

- 202: Returned if the request was successful but waitForSync is false.

- 404: Returned if no graph with this name, no vertex collection or no vertex with this id could be found.

- 412: Returned if if-match header is given, but the documents revision is different.

Examples

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/gharial/social/vertex/female/alice <<EOF

{

"name" : "Alice Cooper",

"age" : 26

}

EOF

HTTP/1.1 202 Accepted

content-type: application/json

etag: 624226536

show response body

Remove a vertex

removes a vertex from a graph

DELETE /_api/gharial/{graph-name}/vertex/{collection-name}/{vertex-key}

Removes a vertex from the collection.

Path Parameters

- graph-name (required): The name of the graph.

- collection-name (required): The name of the vertex collection the vertex belongs to.

- vertex-key (required): The _key attribute of the vertex.

Query Parameters

- waitForSync (optional): Define if the request should wait until synced to disk.

Header Parameters

- if-match (optional): If the "If-Match" header is given, then it must contain exactly one etag. The document is updated, if it has the same revision as the given etag. Otherwise a HTTP 412 is returned. As an alternative you can supply the etag in an attribute rev in the URL.

Return Codes

- 200: Returned if the vertex could be removed.

- 202: Returned if the request was successful but waitForSync is false.

- 404: Returned if no graph with this name, no vertex collection or no vertex with this id could be found.

- 412: Returned if if-match header is given, but the documents revision is different.

Examples

shell> curl -X DELETE --dump - http://localhost:8529/_api/gharial/social/vertex/female/alice

HTTP/1.1 202 Accepted

content-type: application/json

show response body

Handling Edges

Examples will explain the REST API for manipulating edges of the graph module on the knows graph:

Create an edge

Creates an edge in an existing graph

POST /_api/gharial/{graph-name}/edge/{collection-name}

Creates a new edge in the collection. Within the body the has to contain a _from and _to value referencing to valid vertices in the graph. Furthermore the edge has to be valid in the definition of this edge collection.

Path Parameters

- graph-name (required): The name of the graph.

- collection-name (required): The name of the edge collection the edge belongs to.

Query Parameters

- waitForSync (optional): Define if the request should wait until synced to disk.

- _from (required):

- _to (required):

Post Body storeThisJsonObject (required): The body has to be the JSON object to be stored.

Return Codes

- 201: Returned if the edge could be created.

- 202: Returned if the request was successful but waitForSync is false.

- 404: Returned if no graph with this name, no edge collection or no edge with this id could be found.

Examples

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/gharial/social/edge/relation <<EOF

{

"type" : "friend",

"_from" : "female/alice",

"_to" : "female/diana"

}

EOF

HTTP/1.1 202 Accepted

content-type: application/json

etag: 540209384

show response body

Get an edge

fetch an edge

GET /_api/gharial/{graph-name}/edge/{collection-name}/{edge-key}

Gets an edge from the given collection.

Path Parameters

- graph-name (required): The name of the graph.

- collection-name (required): The name of the edge collection the edge belongs to.

- edge-key (required): The _key attribute of the vertex.

Header Parameters

- if-match (optional): If the "If-Match" header is given, then it must contain exactly one etag. The document is returned, if it has the same revision as the given etag. Otherwise a HTTP 412 is returned. As an alternative you can supply the etag in an attribute rev in the URL.

Return Codes

- 200: Returned if the edge could be found.

- 404: Returned if no graph with this name, no edge collection or no edge with this id could be found.

- 412: Returned if if-match header is given, but the documents revision is different.

Examples

shell> curl --dump - http://localhost:8529/_api/gharial/social/edge/relation/aliceAndBob

HTTP/1.1 200 OK

content-type: application/json

etag: 573042920

show response body

Examples will explain the API on the social graph:

Modify an edge

modify an existing edge

PATCH /_api/gharial/{graph-name}/edge/{collection-name}/{edge-key}

Updates the data of the specific edge in the collection.

Path Parameters

- graph-name (required): The name of the graph.

- collection-name (required): The name of the edge collection the edge belongs to.

- edge-key (required): The _key attribute of the vertex.

Query Parameters

- waitForSync (optional): Define if the request should wait until synced to disk.

- keepNull (optional): Define if values set to null should be stored. By default the key is removed from the document.

Post Body updateAttributes (required): The body has to be a JSON object containing the attributes to be updated.

Return Codes

- 200: Returned if the edge could be updated.

- 202: Returned if the request was successful but waitForSync is false.

- 404: Returned if no graph with this name, no edge collection or no edge with this id could be found.

Examples

shell> curl -X PATCH --data-binary @- --dump - http://localhost:8529/_api/gharial/social/edge/relation/aliceAndBob <<EOF

{

"since" : "01.01.2001"

}

EOF

HTTP/1.1 202 Accepted

content-type: application/json

etag: 604369128

show response body

Replace an edge

replace the content of an existing edge

PUT /_api/gharial/{graph-name}/edge/{collection-name}/{edge-key}

Replaces the data of an edge in the collection.

Path Parameters

- graph-name (required): The name of the graph.

- collection-name (required): The name of the edge collection the edge belongs to.

- edge-key (required): The _key attribute of the vertex.

Query Parameters

- waitForSync (optional): Define if the request should wait until synced to disk.

Header Parameters

- if-match (optional): If the "If-Match" header is given, then it must contain exactly one etag. The document is updated, if it has the same revision as the given etag. Otherwise a HTTP 412 is returned. As an alternative you can supply the etag in an attribute rev in the URL.

Post Body storeThisJsonObject (required): The body has to be the JSON object to be stored.

Return Codes

- 200: Returned if the edge could be replaced.

- 202: Returned if the request was successful but waitForSync is false.

- 404: Returned if no graph with this name, no edge collection or no edge with this id could be found.

- 412: Returned if if-match header is given, but the documents revision is different.

Examples

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/gharial/social/edge/relation/aliceAndBob <<EOF

{

"type" : "divorced"

}

EOF

HTTP/1.1 202 Accepted

content-type: application/json

etag: 608235752

show response body

Remove an edge

removes an edge from graph

DELETE /_api/gharial/{graph-name}/edge/{collection-name}/{edge-key}

Removes an edge from the collection.

Path Parameters

- graph-name (required): The name of the graph.

- collection-name (required): The name of the edge collection the edge belongs to.

- edge-key (required): The _key attribute of the vertex.

Query Parameters

- waitForSync (optional): Define if the request should wait until synced to disk.

Header Parameters

- if-match (optional): If the "If-Match" header is given, then it must contain exactly one etag. The document is updated, if it has the same revision as the given etag. Otherwise a HTTP 412 is returned. As an alternative you can supply the etag in an attribute rev in the URL.

Return Codes

- 200: Returned if the edge could be removed.

- 202: Returned if the request was successful but waitForSync is false.

- 404: Returned if no graph with this name, no edge collection or no edge with this id could be found.

- 412: Returned if if-match header is given, but the documents revision is different.

Examples

shell> curl -X DELETE --dump - http://localhost:8529/_api/gharial/social/edge/relation/aliceAndBob

HTTP/1.1 202 Accepted

content-type: application/json

show response body

HTTP Interface for Traversals

Traversals

ArangoDB's graph traversals are executed on the server. Traversals can be initiated by clients by sending the traversal description for execution to the server.

Traversals in ArangoDB are used to walk over a graph stored in one edge collection. It can easily be described which edges of the graph should be followed and which actions should be performed on each visited vertex. Furthermore the ordering of visiting the nodes can be specified, for instance depth-first or breadth-first search are offered.

Executing Traversals via HTTP

executes a traversal

execute a server-side traversal

POST /_api/traversal

Starts a traversal starting from a given vertex and following. edges contained in a given edgeCollection. The request must contain the following attributes.

Body Parameters

- sort: body (JavaScript) code of a custom comparison function for the edges. The signature of this function is (l, r) -> integer (where l and r are edges) and must return -1 if l is smaller than, +1 if l is greater than, and 0 if l and r are equal. The reason for this is the following: The order of edges returned for a certain vertex is undefined. This is because there is no natural order of edges for a vertex with multiple connected edges. To explicitly define the order in which edges on the vertex are followed, you can specify an edge comparator function with this attribute. Note that the value here has to be a string to conform to the JSON standard, which in turn is parsed as function body on the server side. Furthermore note that this attribute is only used for the standard expanders. If you use your custom expander you have to do the sorting yourself within the expander code.

- direction: direction for traversal

- if set, must be either "outbound", "inbound", or "any"

- if not set, the expander attribute must be specified

- maxIterations: Maximum number of iterations in each traversal. This number can be set to prevent endless loops in traversal of cyclic graphs. When a traversal performs as many iterations as the maxIterations value, the traversal will abort with an error. If maxIterations is not set, a server-defined value may be used.

- edgeCollection: name of the collection that contains the edges.

- visitor: body (JavaScript) code of custom visitor function function signature: (config, result, vertex, path, connected) -> void The visitor function can do anything, but its return value is ignored. To populate a result, use the result variable by reference. Note that the connected argument is only populated when the order attribute is set to "preorder-expander".

- itemOrder: item iteration order can be "forward" or "backward"

- order: traversal order can be "preorder", "postorder" or "preorder-expander"

- filter: default is to include all nodes: body (JavaScript code) of custom filter function function signature: (config, vertex, path) -> mixed can return four different string values:

- "exclude" -> this vertex will not be visited.

- "prune" -> the edges of this vertex will not be followed.

- "" or undefined -> visit the vertex and follow it's edges.

- Array -> containing any combination of the above. If there is at least one "exclude" or "prune" respectivly is contained, it's effect will occur.

- init: body (JavaScript) code of custom result initialization function function signature: (config, result) -> void initialize any values in result with what is required

- minDepth: ANDed with any existing filters): visits only nodes in at least the given depth

- maxDepth: ANDed with any existing filters visits only nodes in at most the given depth

- uniqueness: specifies uniqueness for vertices and edges visited if set, must be an object like this: "uniqueness": {"vertices": "none"|"global"|"path", "edges": "none"|"global"|"path"}

- strategy: traversal strategy can be "depthfirst" or "breadthfirst"

- graphName: name of the graph that contains the edges. Either edgeCollection or graphName has to be given. In case both values are set the graphName is prefered.

- expander: body (JavaScript) code of custom expander function must be set if direction attribute is not set function signature: (config, vertex, path) -> array expander must return an array of the connections for vertex each connection is an object with the attributes edge and vertex

- startVertex: id of the startVertex, e.g. "users/foo".

"uniqueness": {"vertices": "none"|"global"|"path", "edges": "none"|"global"|"path"}

If the Traversal is successfully executed HTTP 200 will be returned. Additionally the result object will be returned by the traversal.

For successful traversals, the returned JSON object has the following properties:

- error: boolean flag to indicate if an error occurred (false in this case)

- code: the HTTP status code

- result: the return value of the traversal

If the traversal specification is either missing or malformed, the server will respond with HTTP 400.

The body of the response will then contain a JSON object with additional error details. The object has the following attributes:

- error: boolean flag to indicate that an error occurred (true in this case)

- code: the HTTP status code

- errorNum: the server error number

- errorMessage: a descriptive error message

Return Codes

- 200: If the traversal is fully executed HTTP 200 will be returned.

- 400: If the traversal specification is either missing or malformed, the server will respond with HTTP 400.

- 404: The server will responded with HTTP 404 if the specified edge collection does not exist, or the specified start vertex cannot be found.

- 500: The server will responded with HTTP 500 when an error occurs inside the traversal or if a traversal performs more than maxIterations iterations.

Examples

In the following examples the underlying graph will contain five persons Alice, Bob, Charlie, Dave and Eve. We will have the following directed relations:

- Alice knows Bob

- Bob knows Charlie

- Bob knows Dave

- Eve knows Alice

- Eve knows Bob

The starting vertex will always be Alice.

Follow only outbound edges

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/traversal <<EOF

{

"startVertex" : "persons/alice",

"graphName" : "knows_graph",

"direction" : "outbound"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Follow only inbound edges

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/traversal <<EOF

{

"startVertex" : "persons/alice",

"graphName" : "knows_graph",

"direction" : "inbound"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Follow any direction of edges

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/traversal <<EOF

{

"startVertex" : "persons/alice",

"graphName" : "knows_graph",

"direction" : "any",

"uniqueness" : {

"vertices" : "none",

"edges" : "global"

}

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Excluding Charlie and Bob

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/traversal <<EOF

{

"startVertex" : "persons/alice",

"graphName" : "knows_graph",

"direction" : "outbound",

"filter" : "if (vertex.name === \"Bob\" ||     vertex.name === \"Charlie\") {  return \"exclude\";}return;"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Do not follow edges from Bob

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/traversal <<EOF

{

"startVertex" : "persons/alice",

"graphName" : "knows_graph",

"direction" : "outbound",

"filter" : "if (vertex.name === \"Bob\") {return \"prune\";}return;"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Visit only nodes in a depth of at least 2

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/traversal <<EOF

{

"startVertex" : "persons/alice",

"graphName" : "knows_graph",

"direction" : "outbound",

"minDepth" : 2

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Visit only nodes in a depth of at most 1

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/traversal <<EOF

{

"startVertex" : "persons/alice",

"graphName" : "knows_graph",

"direction" : "outbound",

"maxDepth" : 1

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Using a visitor function to return vertex ids only

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/traversal <<EOF

{

"startVertex" : "persons/alice",

"graphName" : "knows_graph",

"direction" : "outbound",

"visitor" : "result.visited.vertices.push(vertex._id);"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Count all visited nodes and return a list of nodes only

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/traversal <<EOF

{

"startVertex" : "persons/alice",

"graphName" : "knows_graph",

"direction" : "outbound",

"init" : "result.visited = 0; result.myVertices = [ ];",

"visitor" : "result.visited++; result.myVertices.push(vertex);"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Expand only inbound edges of Alice and outbound edges of Eve

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/traversal <<EOF

{

"startVertex" : "persons/alice",

"graphName" : "knows_graph",

"expander" : "var connections = [ ];if (vertex.name === \"Alice\") {config.datasource.getInEdges(vertex).forEach(function (e) {connections.push({ vertex: require(\"internal\").db._document(e._from), edge: e});});}if (vertex.name === \"Eve\") {config.datasource.getOutEdges(vertex).forEach(function (e) {connections.push({vertex: require(\"internal\").db._document(e._to), edge: e});});}return connections;"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Follow the depthfirst strategy

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/traversal <<EOF

{

"startVertex" : "persons/alice",

"graphName" : "knows_graph",

"direction" : "any",

"strategy" : "depthfirst"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Using postorder ordering

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/traversal <<EOF

{

"startVertex" : "persons/alice",

"graphName" : "knows_graph",

"direction" : "any",

"order" : "postorder"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Using backward item-ordering:

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/traversal <<EOF

{

"startVertex" : "persons/alice",

"graphName" : "knows_graph",

"direction" : "any",

"itemOrder" : "backward"

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Edges should only be included once globally, but nodes are included every time they are visited

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/traversal <<EOF

{

"startVertex" : "persons/alice",

"graphName" : "knows_graph",

"direction" : "any",

"uniqueness" : {

"vertices" : "none",

"edges" : "global"

}

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

If the underlying graph is cyclic, maxIterations should be set

The underlying graph has two vertices Alice and Bob. With the directed edges:

- Alice knows Bob _ Bob knows Alice

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/traversal <<EOF

{

"startVertex" : "persons/alice",

"graphName" : "knows_graph",

"direction" : "any",

"uniqueness" : {

"vertices" : "none",

"edges" : "none"

},

"maxIterations" : 5

}

EOF

HTTP/1.1 500 Internal Server Error

content-type: application/json; charset=utf-8

show response body

All examples were using this graph:

HTTP Interface for Replication

Replication

This is an introduction to ArangoDB's HTTP replication interface. The replication architecture and components are described in more details in Replication.

The HTTP replication interface serves four main purposes:

- fetch initial data from a server (e.g. for a backup, or for the initial synchronization of data before starting the continuous replication applier)

- querying the state of a master

- fetch continuous changes from a master (used for incremental synchronization of changes)

- administer the replication applier (starting, stopping, configuring, querying state) on a slave

Please note that all replication operations work on a per-database level. If an ArangoDB server contains more than one database, the replication system must be configured individually per database, and replicating the data of multiple databases will require multiple operations.

Replication Dump Commands

The inventory method can be used to query an ArangoDB database's current set of collections plus their indexes. Clients can use this method to get an overview of which collections are present in the database. They can use this information to either start a full or a partial synchronization of data, e.g. to initiate a backup or the incremental data synchronization.

Return inventory of collections and indexes

Returns an overview of collections and their indexes

GET /_api/replication/inventory

Query Parameters

- includeSystem (optional): Include system collections in the result. The default value is true.

Returns the array of collections and indexes available on the server. This array can be used by replication clients to initiate an initial sync with the server.

The response will contain a JSON object with the collection and state and tick attributes.

collections is a array of collections with the following sub-attributes:

- parameters: the collection properties

- indexes: a array of the indexes of a the collection. Primary indexes and edges indexes are not included in this array.

The state attribute contains the current state of the replication logger. It contains the following sub-attributes:

- running: whether or not the replication logger is currently active. Note: since ArangoDB 2.2, the value will always be true

- lastLogTick: the value of the last tick the replication logger has written

- time: the current time on the server

Replication clients should note the lastLogTick value returned. They can then fetch collections' data using the dump method up to the value of lastLogTick, and query the continuous replication log for log events after this tick value.

To create a full copy of the collections on the server, a replication client can execute these steps:

- call the /inventory API method. This returns the lastLogTick value and the array of collections and indexes from the server.

- for each collection returned by /inventory, create the collection locally and call /dump to stream the collection data to the client, up to the value of lastLogTick. After that, the client can create the indexes on the collections as they were reported by /inventory.

If the clients wants to continuously stream replication log events from the logger server, the following additional steps need to be carried out:

- the client should call /logger-follow initially to fetch the first batch of replication events that were logged after the client's call to /inventory.

The call to /logger-follow should use a from parameter with the value of the lastLogTick as reported by /inventory. The call to /logger-follow will return the x-arango-replication-lastincluded which will contain the last tick value included in the response.

- the client can then continuously call /logger-follow to incrementally fetch new replication events that occurred after the last transfer.

Calls should use a from parameter with the value of the x-arango-replication-lastincluded header of the previous response. If there are no more replication events, the response will be empty and clients can go to sleep for a while and try again later.

Note: on a coordinator, this request must have the query parameter DBserver which must be an ID of a DBserver. The very same request is forwarded synchronously to that DBserver. It is an error if this attribute is not bound in the coordinator case.

Return Codes

- 200: is returned if the request was executed successfully.

- 405: is returned when an invalid HTTP method is used.

- 500: is returned if an error occurred while assembling the response.

Examples

shell> curl --dump - http://localhost:8529/_api/replication/inventory

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

With some additional indexes:

shell> curl --dump - http://localhost:8529/_api/replication/inventory

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

The dump method can be used to fetch data from a specific collection. As the results of the dump command can be huge, dump may not return all data from a collection at once. Instead, the dump command may be called repeatedly by replication clients until there is no more data to fetch. The dump command will not only return the current documents in the collection, but also document updates and deletions.

Please note that the dump method will only return documents, updates and deletions from a collection's journals and datafiles. Operations that are stored in the write-ahead log only will not be returned. In order to ensure that these operations are included in a dump, the write-ahead log must be flushed first.

To get to an identical state of data, replication clients should apply the individual parts of the dump results in the same order as they are provided.

Return data of a collection

returns the whole content of one collection

GET /_api/replication/dump

Query Parameters

- collection (required): The name or id of the collection to dump.

- from (optional): Lower bound tick value for results.

- to (optional): Upper bound tick value for results.

- chunkSize (optional): Approximate maximum size of the returned result.

- includeSystem (optional): Include system collections in the result. The default value is true.

- failOnUnknown (optional): Produce an error when dumped edges refer to now-unknown collections.

- ticks (optional): Whether or not to include tick values in the dump. The default value is true.

- flush (optional): Whether or not to flush the WAL before dumping. The default value is true.

Returns the data from the collection for the requested range.

When the from query parameter is not used, collection events are returned from the beginning. When the from parameter is used, the result will only contain collection entries which have higher tick values than the specified from value (note: the log entry with a tick value equal to from will be excluded).

The to query parameter can be used to optionally restrict the upper bound of the result to a certain tick value. If used, the result will only contain collection entries with tick values up to (including) to.

The chunkSize query parameter can be used to control the size of the result. It must be specified in bytes. The chunkSize value will only be honored approximately. Otherwise a too low chunkSize value could cause the server to not be able to put just one entry into the result and return it. Therefore, the chunkSize value will only be consulted after an entry has been written into the result. If the result size is then bigger than chunkSize, the server will respond with as many entries as there are in the response already. If the result size is still smaller than chunkSize, the server will try to return more data if there's more data left to return.

If chunkSize is not specified, some server-side default value will be used.

The Content-Type of the result is application/x-arango-dump. This is an easy-to-process format, with all entries going onto separate lines in the response body.

Each line itself is a JSON object, with at least the following attributes:

- tick: the operation's tick attribute

- key: the key of the document/edge or the key used in the deletion operation

- rev: the revision id of the document/edge or the deletion operation

- data: the actual document/edge data for types 2300 and 2301. The full document/edge data will be returned even for updates.

- type: the type of entry. Possible values for type are:

- 2300: document insertion/update

- 2301: edge insertion/update

- 2302: document/edge deletion

Note: there will be no distinction between inserts and updates when calling this method.

Return Codes

- 200: is returned if the request was executed successfully and data was returned. The header  x-arango-replication-lastincluded  is set to the tick of the last document returned.

- 204: is returned if the request was executed successfully, but there was no content available. The header  x-arango-replication-lastincluded  is  0  in this case.

- 400: is returned if either the from or to values are invalid.

- 404: is returned when the collection could not be found.

- 405: is returned when an invalid HTTP method is used.

- 500: is returned if an error occurred while assembling the response.

Examples

Empty collection:

shell> curl --dump - http://localhost:8529/_api/replication/dump?collection=testCollection

HTTP/1.1 204 No Content

content-type: application/x-arango-dump; charset=utf-8

x-arango-replication-checkmore: false

x-arango-replication-lastincluded: 0

Non-empty collection:

shell> curl --dump - http://localhost:8529/_api/replication/dump?collection=testCollection

HTTP/1.1 200 OK

content-type: application/x-arango-dump; charset=utf-8

x-arango-replication-checkmore: false

x-arango-replication-lastincluded: 788001000

"{\"tick\":\"787345640\",\"type\":2300,\"key\":\"123456\",\"rev\":\"787280104\",\"data\":{\"_key\":\"123456\",\"_rev\":\"787280104\",\"c\":false,\"b\":1,\"d\":\"additional value\"}}

{\"tick\":\"787804392\",\"type\":2302,\"key\":\"foobar\",\"rev\":\"787738856\"}

{\"tick\":\"788001000\",\"type\":2302,\"key\":\"abcdef\",\"rev\":\"787935464\"}

"

Synchronize data from a remote endpoint

start a replication

PUT /_api/replication/sync

Body Parameters

- username: an optional ArangoDB username to use when connecting to the endpoint.

- includeSystem: whether or not system collection operations will be applied

- endpoint: the master endpoint to connect to (e.g. "tcp://192.168.173.13:8529").

- initialSyncMaxWaitTime: the maximum wait time (in seconds) that the initial synchronization will wait for a response from the master when fetching initial collection data. This wait time can be used to control after what time the initial synchronization will give up waiting for a response and fail. This value will be ignored if set to 0.

- database: the database name on the master (if not specified, defaults to the name of the local current database).

- restrictType: an optional string value for collection filtering. When specified, the allowed values are include or exclude.

- incremental: if set to true, then an incremental synchronization method will be used for synchronizing data in collections. This method is useful when collections already exist locally, and only the remaining differences need to be transferred from the remote endpoint. In this case, the incremental synchronization can be faster than a full synchronization. The default value is false, meaning that the complete data from the remote collection will be transferred.

- restrictCollections: an optional array of collections for use with restrictType. If restrictType is include, only the specified collections will be sychronised. If restrictType is exclude, all but the specified collections will be synchronized. of type string

- password: the password to use when connecting to the endpoint.

Starts a full data synchronization from a remote endpoint into the local ArangoDB database.

The sync method can be used by replication clients to connect an ArangoDB database to a remote endpoint, fetch the remote list of collections and indexes, and collection data. It will thus create a local backup of the state of data at the remote ArangoDB database. sync works on a per-database level.

sync will first fetch the list of collections and indexes from the remote endpoint. It does so by calling the inventory API of the remote database. It will then purge data in the local ArangoDB database, and after start will transfer collection data from the remote database to the local ArangoDB database. It will extract data from the remote database by calling the remote database's dump API until all data are fetched.

In case of success, the body of the response is a JSON object with the following attributes:

- collections: an array of collections that were transferred from the endpoint

- lastLogTick: the last log tick on the endpoint at the time the transfer was started. Use this value as the from value when starting the continuous synchronization later.

WARNING: calling this method will sychronize data from the collections found on the remote endpoint to the local ArangoDB database. All data in the local collections will be purged and replaced with data from the endpoint.

Use with caution!

Note: this method is not supported on a coordinator in a cluster.

Return Codes

- 200: is returned if the request was executed successfully.

- 400: is returned if the configuration is incomplete or malformed.

- 405: is returned when an invalid HTTP method is used.

- 500: is returned if an error occurred during sychronization.

- 501: is returned when this operation is called on a coordinator in a cluster.

Return cluster inventory of collections and indexes

returs an overview of collections and indexes in a cluster

GET /_api/replication/clusterInventory

Query Parameters

- includeSystem (optional): Include system collections in the result. The default value is true.

Returns the array of collections and indexes available on the cluster.

The response will be an array of JSON objects, one for each collection. Each collection containscontains exactly two keys "parameters" and "indexes". This information comes from Plan/Collections/{DB-Name}/ in the agency, just that the indexes* attribute there is relocated to adjust it to the data format of arangodump.

Return Codes

- 200: is returned if the request was executed successfully.

- 405: is returned when an invalid HTTP method is used.

- 500: is returned if an error occurred while assembling the response.

Replication Logger Commands

Previous versions of ArangoDB allowed starting, stopping and configuring the replication logger. These commands are superfluous in ArangoDB 2.2 as all data-modification operations are written to the server's write-ahead log and are not handled by a separate logger anymore.

The only useful operations remaining since ArangoDB 2.2 are to query the current state of the logger and to fetch the latest changes written by the logger. The operations will return the state and data from the write-ahead log.

Return replication logger state

returns the state of the replication logger

GET /_api/replication/logger-state

Returns the current state of the server's replication logger. The state will include information about whether the logger is running and about the last logged tick value. This tick value is important for incremental fetching of data.

The body of the response contains a JSON object with the following attributes:

- state: the current logger state as a JSON object with the following sub-attributes:

- running: whether or not the logger is running

- lastLogTick: the tick value of the latest tick the logger has logged. This value can be used for incremental fetching of log data.

- totalEvents: total number of events logged since the server was started. The value is not reset between multiple stops and re-starts of the logger.

- time: the current date and time on the logger server

- server: a JSON object with the following sub-attributes:

- version: the logger server's version

- serverId: the logger server's id

- clients: returns the last fetch status by replication clients connected to the logger. Each client is returned as a JSON object with the following attributes:

- serverId: server id of client

- lastServedTick: last tick value served to this client via the logger-follow API

- time: date and time when this client last called the logger-follow API

Return Codes

- 200: is returned if the logger state could be determined successfully.

- 405: is returned when an invalid HTTP method is used.

- 500: is returned if the logger state could not be determined.

Examples

Returns the state of the replication logger.

shell> curl --dump - http://localhost:8529/_api/replication/logger-state

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

To query the latest changes logged by the replication logger, the HTTP interface also provides the  logger-follow  method.

This method should be used by replication clients to incrementally fetch updates from an ArangoDB database.

Returns log entries

Fetch log lines from the server

GET /_api/replication/logger-follow

Query Parameters

- from (optional): Lower bound tick value for results.

- to (optional): Upper bound tick value for results.

- chunkSize (optional): Approximate maximum size of the returned result.

- includeSystem (optional): Include system collections in the result. The default value is true.

Returns data from the server's replication log. This method can be called by replication clients after an initial synchronization of data. The method will return all "recent" log entries from the logger server, and the clients can replay and apply these entries locally so they get to the same data state as the logger server.

Clients can call this method repeatedly to incrementally fetch all changes from the logger server. In this case, they should provide the from value so they will only get returned the log events since their last fetch.

When the from query parameter is not used, the logger server will return log entries starting at the beginning of its replication log. When the from parameter is used, the logger server will only return log entries which have higher tick values than the specified from value (note: the log entry with a tick value equal to from will be excluded). Use the from value when incrementally fetching log data.

The to query parameter can be used to optionally restrict the upper bound of the result to a certain tick value. If used, the result will contain only log events with tick values up to (including) to. In incremental fetching, there is no need to use the to parameter. It only makes sense in special situations, when only parts of the change log are required.

The chunkSize query parameter can be used to control the size of the result. It must be specified in bytes. The chunkSize value will only be honored approximately. Otherwise a too low chunkSize value could cause the server to not be able to put just one log entry into the result and return it. Therefore, the chunkSize value will only be consulted after a log entry has been written into the result. If the result size is then bigger than chunkSize, the server will respond with as many log entries as there are in the response already. If the result size is still smaller than chunkSize, the server will try to return more data if there's more data left to return.

If chunkSize is not specified, some server-side default value will be used.

The Content-Type of the result is application/x-arango-dump. This is an easy-to-process format, with all log events going onto separate lines in the response body. Each log event itself is a JSON object, with at least the following attributes:

- tick: the log event tick value

- type: the log event type

Individual log events will also have additional attributes, depending on the event type. A few common attributes which are used for multiple events types are:

- cid: id of the collection the event was for

- tid: id of the transaction the event was contained in

- key: document key

- rev: document revision id

- data: the original document data

A more detailed description of the individual replication event types and their data structures can be found in @ref RefManualReplicationEventTypes.

The response will also contain the following HTTP headers:

- x-arango-replication-active: whether or not the logger is active. Clients can use this flag as an indication for their polling frequency. If the logger is not active and there are no more replication events available, it might be sensible for a client to abort, or to go to sleep for a long time and try again later to check whether the logger has been activated.

- x-arango-replication-lastincluded: the tick value of the last included value in the result. In incremental log fetching, this value can be used as the from value for the following request. Note that if the result is empty, the value will be 0. This value should not be used as from value by clients in the next request (otherwise the server would return the log events from the start of the log again).

- x-arango-replication-lasttick: the last tick value the logger server has logged (not necessarily included in the result). By comparing the the last tick and last included tick values, clients have an approximate indication of how many events there are still left to fetch.

- x-arango-replication-checkmore: whether or not there already exists more log data which the client could fetch immediately. If there is more log data available, the client could call logger-follow again with an adjusted from value to fetch remaining log entries until there are no more.

If there isn't any more log data to fetch, the client might decide to go to sleep for a while before calling the logger again.

Note: this method is not supported on a coordinator in a cluster.

Return Codes

- 200: is returned if the request was executed successfully, and there are log events available for the requested range. The response body will not be empty in this case.

- 204: is returned if the request was executed successfully, but there are no log events available for the requested range. The response body will be empty in this case.

- 400: is returned if either the from or to values are invalid.

- 405: is returned when an invalid HTTP method is used.

- 500: is returned if an error occurred while assembling the response.

- 501: is returned when this operation is called on a coordinator in a cluster.

Examples

No log events available

shell> curl --dump - http://localhost:8529/_api/replication/logger-follow?from=791933160

HTTP/1.1 204 No Content

content-type: application/x-arango-dump; charset=utf-8

x-arango-replication-active: true

x-arango-replication-checkmore: false

x-arango-replication-frompresent: true

x-arango-replication-lastincluded: 0

x-arango-replication-lasttick: 791933160

A few log events

shell> curl --dump - http://localhost:8529/_api/replication/logger-follow?from=791933160

HTTP/1.1 200 OK

content-type: application/x-arango-dump; charset=utf-8

x-arango-replication-active: true

x-arango-replication-checkmore: false

x-arango-replication-frompresent: true

x-arango-replication-lastincluded: 793178344

x-arango-replication-lasttick: 793178344

"{\"tick\":\"792064232\",\"type\":2000,\"database\":\"127208\",\"cid\":\"791998696\",\"collection\":{\"version\":5,\"type\":2,\"cid\":\"791998696\",\"indexBuckets\":8,\"deleted\":false,\"doCompact\":true,\"maximalSize\":1048576,\"name\":\"products\",\"isVolatile\":false,\"waitForSync\":false}}

{\"tick\":\"792391912\",\"type\":2300,\"database\":\"127208\",\"cid\":\"791998696\",\"tid\":\"0\",\"key\":\"p1\",\"rev\":\"792326376\",\"data\":{\"_key\":\"p1\",\"_rev\":\"792326376\",\"name\":\"flux compensator\"}}

{\"tick\":\"792719592\",\"type\":2300,\"database\":\"127208\",\"cid\":\"791998696\",\"tid\":\"0\",\"key\":\"p2\",\"rev\":\"792654056\",\"data\":{\"_key\":\"p2\",\"_rev\":\"792654056\",\"hp\":5100,\"name\":\"hybrid hovercraft\"}}

{\"tick\":\"792916200\",\"type\":2302,\"database\":\"127208\",\"cid\":\"791998696\",\"tid\":\"0\",\"key\":\"p1\",\"rev\":\"792850664\"}

{\"tick\":\"793112808\",\"type\":2300,\"database\":\"127208\",\"cid\":\"791998696\",\"tid\":\"0\",\"key\":\"p2\",\"rev\":\"793047272\",\"data\":{\"_key\":\"p2\",\"_rev\":\"793047272\"}}

{\"tick\":\"793178344\",\"type\":2001,\"database\":\"127208\",\"cid\":\"791998696\"}

"

More events than would fit into the response

shell> curl --dump - http://localhost:8529/_api/replication/logger-follow?from=788590824&chunkSize=400

HTTP/1.1 200 OK

content-type: application/x-arango-dump; charset=utf-8

x-arango-replication-active: true

x-arango-replication-checkmore: true

x-arango-replication-frompresent: true

x-arango-replication-lastincluded: 789049576

x-arango-replication-lasttick: 791933160

"{\"tick\":\"788656360\",\"type\":2001,\"database\":\"127208\",\"cid\":\"788328680\"}

{\"tick\":\"788787432\",\"type\":2000,\"database\":\"127208\",\"cid\":\"788721896\",\"collection\":{\"version\":5,\"type\":2,\"cid\":\"788721896\",\"indexBuckets\":8,\"deleted\":false,\"doCompact\":true,\"maximalSize\":1048576,\"name\":\"IndexedCollection1\",\"isVolatile\":false,\"waitForSync\":false}}

{\"tick\":\"789049576\",\"type\":2100,\"database\":\"127208\",\"cid\":\"788721896\",\"id\":\"788984040\",\"index\":{\"id\":\"788984040\",\"type\":\"hash\",\"fields\":[\"name\"],\"selectivityEstimate\":1,\"unique\":false,\"sparse\":false}}

"

To check what range of changes is available (identified by tick values), the HTTP interface provides the methods  logger-first-tick  and  logger-tick-ranges . Replication clients can use the methods to determine if certain data (identified by a tick date) is still available on the master.

Returns the first available tick value

Return the first available tick value from the server

GET /_api/replication/logger-first-tick

Returns the first available tick value that can be served from the server's replication log. This method can be called by replication clients after to determine if certain data (identified by a tick value) is still available for replication.

The result is a JSON object containing the attribute firstTick. This attribute contains the minimum tick value available in the server's replication log.

Note: this method is not supported on a coordinator in a cluster.

Return Codes

- 200: is returned if the request was executed successfully.

- 405: is returned when an invalid HTTP method is used.

- 500: is returned if an error occurred while assembling the response.

- 501: is returned when this operation is called on a coordinator in a cluster.

Examples

Returning the first available tick

shell> curl --dump - http://localhost:8529/_api/replication/logger-first-tick

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

"{\"firstTick\":\"389352\"}"

Return the tick ranges available in the WAL logfiles

returns the tick value ranges available in the logfiles

GET /_api/replication/logger-tick-ranges

Returns the currently available ranges of tick values for all currently available WAL logfiles. The tick values can be used to determine if certain data (identified by tick value) are still available for replication.

The body of the response contains a JSON array. Each array member is an object that describes a single logfile. Each object has the following attributes:

- datafile: name of the logfile

- status: status of the datafile, in textual form (e.g. "sealed", "open")

- tickMin: minimum tick value contained in logfile

- tickMax: maximum tick value contained in logfile

Return Codes

- 200: is returned if the tick ranges could be determined successfully.

- 405: is returned when an invalid HTTP method is used.

- 500: is returned if the logger state could not be determined.

- 501: is returned when this operation is called on a coordinator in a cluster.

Examples

Returns the available tick ranges.

shell> curl --dump - http://localhost:8529/_api/replication/logger-tick-ranges

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

[

{

"datafile" : "/tmp/vocdir.26839/journals/logfile-192744.db",

"status" : "collected",

"tickMin" : "389352",

"tickMax" : "666300648"

},

{

"datafile" : "/tmp/vocdir.26839/journals/logfile-666431720.db",

"status" : "collected",

"tickMin" : "666759400",

"tickMax" : "669184232"

},

{

"datafile" : "/tmp/vocdir.26839/journals/logfile-669315304.db",

"status" : "collected",

"tickMin" : "669446376",

"tickMax" : "788001000"

},

{

"datafile" : "/tmp/vocdir.26839/journals/logfile-788132072.db",

"status" : "collected",

"tickMin" : "788263144",

"tickMax" : "788394216"

},

{

"datafile" : "/tmp/vocdir.26839/journals/logfile-788525288.db",

"status" : "open",

"tickMin" : "788656360",

"tickMax" : "793178344"

}

]

Replication Applier Commands

The applier commands allow to remotely start, stop, and query the state and configuration of an ArangoDB database's replication applier.

Return configuration of replication applier

fetch the current replication configuration

GET /_api/replication/applier-config

Returns the configuration of the replication applier.

The body of the response is a JSON object with the configuration. The following attributes may be present in the configuration:

- endpoint: the logger server to connect to (e.g. "tcp://192.168.173.13:8529").

- database: the name of the database to connect to (e.g. "_system").

- username: an optional ArangoDB username to use when connecting to the endpoint.

- password: the password to use when connecting to the endpoint.

- maxConnectRetries: the maximum number of connection attempts the applier will make in a row. If the applier cannot establish a connection to the endpoint in this number of attempts, it will stop itself.

- connectTimeout: the timeout (in seconds) when attempting to connect to the endpoint. This value is used for each connection attempt.

- requestTimeout: the timeout (in seconds) for individual requests to the endpoint.

- chunkSize: the requested maximum size for log transfer packets that is used when the endpoint is contacted.

- autoStart: whether or not to auto-start the replication applier on (next and following) server starts

- adaptivePolling: whether or not the replication applier will use adaptive polling.

- includeSystem: whether or not system collection operations will be applied

- autoResync: whether or not the slave should perform a full automatic resynchronization with the master in case the master cannot serve log data requested by the slave, or when the replication is started and no tick value can be found.

- autoResyncRetries: number of resynchronization retries that will be performed in a row when automatic resynchronization is enabled and kicks in. Setting this to 0 will effectively disable autoResync. Setting it to some other value will limit the number of retries that are performed. This helps preventing endless retries in case resynchronizations always fail.

- initialSyncMaxWaitTime: the maximum wait time (in seconds) that the initial synchronization will wait for a response from the master when fetching initial collection data. This wait time can be used to control after what time the initial synchronization will give up waiting for a response and fail. This value is relevant even for continuous replication when autoResync is set to true because this may re-start the initial synchronization when the master cannot provide log data the slave requires. This value will be ignored if set to 0.

- connectionRetryWaitTime: the time (in seconds) that the applier will intentionally idle before it retries connecting to the master in case of connection problems. This value will be ignored if set to 0.

- idleMinWaitTime: the minimum wait time (in seconds) that the applier will intentionally idle before fetching more log data from the master in case the master has already sent all its log data. This wait time can be used to control the frequency with which the replication applier sends HTTP log fetch requests to the master in case there is no write activity on the master. This value will be ignored if set to 0.

- idleMaxWaitTime: the maximum wait time (in seconds) that the applier will intentionally idle before fetching more log data from the master in case the master has already sent all its log data and there have been previous log fetch attempts that resulted in no more log data. This wait time can be used to control the maximum frequency with which the replication applier sends HTTP log fetch requests to the master in case there is no write activity on the master for longer periods. This configuration value will only be used if the option adaptivePolling is set to true. This value will be ignored if set to 0.

- requireFromPresent: if set to true, then the replication applier will check at start whether the start tick from which it starts or resumes replication is still present on the master. If not, then there would be data loss. If requireFromPresent is true, the replication applier will abort with an appropriate error message. If set to false, then the replication applier will still start, and ignore the data loss.

- verbose: if set to true, then a log line will be emitted for all operations performed by the replication applier. This should be used for debugging replication problems only.

- restrictType: the configuration for restrictCollections

- restrictCollections: the optional array of collections to include or exclude, based on the setting of restrictType

Return Codes

- 200: is returned if the request was executed successfully.

- 405: is returned when an invalid HTTP method is used.

- 500: is returned if an error occurred while assembling the response.

Examples

shell> curl --dump - http://localhost:8529/_api/replication/applier-config

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Adjust configuration of replication applier

set configuration values of an applier

PUT /_api/replication/applier-config

Body Parameters

- username: an optional ArangoDB username to use when connecting to the endpoint.

- includeSystem: whether or not system collection operations will be applied

- endpoint: the logger server to connect to (e.g. "tcp://192.168.173.13:8529"). The endpoint must be specified.

- verbose: if set to true, then a log line will be emitted for all operations performed by the replication applier. This should be used for debugging replication problems only.

- connectTimeout: the timeout (in seconds) when attempting to connect to the endpoint. This value is used for each connection attempt.

- chunkSize: the requested maximum size for log transfer packets that is used when the endpoint is contacted.

- idleMinWaitTime: the minimum wait time (in seconds) that the applier will intentionally idle before fetching more log data from the master in case the master has already sent all its log data. This wait time can be used to control the frequency with which the replication applier sends HTTP log fetch requests to the master in case there is no write activity on the master. This value will be ignored if set to 0.

- requestTimeout: the timeout (in seconds) for individual requests to the endpoint.

- database: the name of the database on the endpoint. If not specified, defaults to the current local database name.

- requireFromPresent: if set to true, then the replication applier will check at start whether the start tick from which it starts or resumes replication is still present on the master. If not, then there would be data loss. If requireFromPresent is true, the replication applier will abort with an appropriate error message. If set to false, then the replication applier will still start, and ignore the data loss.

- restrictType: the configuration for restrictCollections; Has to be either include or exclude

- password: the password to use when connecting to the endpoint.

- idleMaxWaitTime: the maximum wait time (in seconds) that the applier will intentionally idle before fetching more log data from the master in case the master has already sent all its log data and there have been previous log fetch attempts that resulted in no more log data. This wait time can be used to control the maximum frequency with which the replication applier sends HTTP log fetch requests to the master in case there is no write activity on the master for longer periods. This configuration value will only be used if the option adaptivePolling is set to true. This value will be ignored if set to 0.

- adaptivePolling: if set to true, the replication applier will fall to sleep for an increasingly long period in case the logger server at the endpoint does not have any more replication events to apply. Using adaptive polling is thus useful to reduce the amount of work for both the applier and the logger server for cases when there are only infrequent changes. The downside is that when using adaptive polling, it might take longer for the replication applier to detect that there are new replication events on the logger server. Setting adaptivePolling to false will make the replication applier contact the logger server in a constant interval, regardless of whether the logger server provides updates frequently or seldom.

- autoStart: whether or not to auto-start the replication applier on (next and following) server starts

- restrictCollections: the array of collections to include or exclude, based on the setting of restrictType of type string

- maxConnectRetries: the maximum number of connection attempts the applier will make in a row. If the applier cannot establish a connection to the endpoint in this number of attempts, it will stop itself.

- initialSyncMaxWaitTime: the maximum wait time (in seconds) that the initial synchronization will wait for a response from the master when fetching initial collection data. This wait time can be used to control after what time the initial synchronization will give up waiting for a response and fail. This value is relevant even for continuous replication when autoResync is set to true because this may re-start the initial synchronization when the master cannot provide log data the slave requires. This value will be ignored if set to 0.

- connectionRetryWaitTime: the time (in seconds) that the applier will intentionally idle before it retries connecting to the master in case of connection problems. This value will be ignored if set to 0.

- autoResyncRetries: number of resynchronization retries that will be performed in a row when automatic resynchronization is enabled and kicks in. Setting this to 0 will effectively disable autoResync. Setting it to some other value will limit the number of retries that are performed. This helps preventing endless retries in case resynchronizations always fail.

- autoResync: whether or not the slave should perform a full automatic resynchronization with the master in case the master cannot serve log data requested by the slave, or when the replication is started and no tick value can be found.

Setting adaptivePolling to false will make the replication applier contact the logger server in a constant interval, regardless of whether the logger server provides updates frequently or seldom.

Sets the configuration of the replication applier. The configuration can only be changed while the applier is not running. The updated configuration will be saved immediately but only become active with the next start of the applier.

In case of success, the body of the response is a JSON object with the updated configuration.

Return Codes

- 200: is returned if the request was executed successfully.

- 400: is returned if the configuration is incomplete or malformed, or if the replication applier is currently running.

- 405: is returned when an invalid HTTP method is used.

- 500: is returned if an error occurred while assembling the response.

Examples

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/replication/applier-config <<EOF

{

"endpoint" : "tcp://127.0.0.1:8529",

"username" : "replicationApplier",

"password" : "applier1234@foxx",

"chunkSize" : 4194304,

"autoStart" : false,

"adaptivePolling" : true

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Start replication applier

start the replication applier

PUT /_api/replication/applier-start

Query Parameters

- from (optional): The remote lastLogTick value from which to start applying. If not specified, the last saved tick from the previous applier run is used. If there is no previous applier state saved, the applier will start at the beginning of the logger server's log.

Starts the replication applier. This will return immediately if the replication applier is already running.

If the replication applier is not already running, the applier configuration will be checked, and if it is complete, the applier will be started in a background thread. This means that even if the applier will encounter any errors while running, they will not be reported in the response to this method.

To detect replication applier errors after the applier was started, use the /_api/replication/applier-state API instead.

Return Codes

- 200: is returned if the request was executed successfully.

- 400: is returned if the replication applier is not fully configured or the configuration is invalid.

- 405: is returned when an invalid HTTP method is used.

- 500: is returned if an error occurred while assembling the response.

Examples

shell> curl -X PUT --dump - http://localhost:8529/_api/replication/applier-start

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Stop replication applier

stop the replication

PUT /_api/replication/applier-stop

Stops the replication applier. This will return immediately if the replication applier is not running.

Return Codes

- 200: is returned if the request was executed successfully.

- 405: is returned when an invalid HTTP method is used.

- 500: is returned if an error occurred while assembling the response.

Examples

shell> curl -X PUT --dump - http://localhost:8529/_api/replication/applier-stop

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

State of the replication applier

output the current status of the replication

GET /_api/replication/applier-state

Returns the state of the replication applier, regardless of whether the applier is currently running or not.

The response is a JSON object with the following attributes:

- state: a JSON object with the following sub-attributes:

- running: whether or not the applier is active and running

- lastAppliedContinuousTick: the last tick value from the continuous replication log the applier has applied.

- lastProcessedContinuousTick: the last tick value from the continuous replication log the applier has processed.

Regularly, the last applied and last processed tick values should be identical. For transactional operations, the replication applier will first process incoming log events before applying them, so the processed tick value might be higher than the applied tick value. This will be the case until the applier encounters the transaction commit log event for the transaction.

- lastAvailableContinuousTick: the last tick value the logger server can provide.

- time: the time on the applier server.

- totalRequests: the total number of requests the applier has made to the endpoint.

- totalFailedConnects: the total number of failed connection attempts the applier has made.

- totalEvents: the total number of log events the applier has processed.

- totalOperationsExcluded: the total number of log events excluded because of restrictCollections.

- progress: a JSON object with details about the replication applier progress. It contains the following sub-attributes if there is progress to report:

- message: a textual description of the progress

- time: the date and time the progress was logged

- failedConnects: the current number of failed connection attempts

- lastError: a JSON object with details about the last error that happened on the applier. It contains the following sub-attributes if there was an error:

- errorNum: a numerical error code

- errorMessage: a textual error description

- time: the date and time the error occurred

In case no error has occurred, lastError will be empty.

- server: a JSON object with the following sub-attributes:

- version: the applier server's version

- serverId: the applier server's id

- endpoint: the endpoint the applier is connected to (if applier is active) or will connect to (if applier is currently inactive)

- database: the name of the database the applier is connected to (if applier is active) or will connect to (if applier is currently inactive)

Return Codes

- 200: is returned if the request was executed successfully.

- 405: is returned when an invalid HTTP method is used.

- 500: is returned if an error occurred while assembling the response.

Examples

Fetching the state of an inactive applier:

shell> curl --dump - http://localhost:8529/_api/replication/applier-state

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Fetching the state of an active applier:

shell> curl --dump - http://localhost:8529/_api/replication/applier-state

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Turn the server into a slave of another

Changes role to slave

PUT /_api/replication/make-slave

Body Parameters

- username: an optional ArangoDB username to use when connecting to the master.

- includeSystem: whether or not system collection operations will be applied

- endpoint: the master endpoint to connect to (e.g. "tcp://192.168.173.13:8529").

- verbose: if set to true, then a log line will be emitted for all operations performed by the replication applier. This should be used for debugging replication problems only.

- connectTimeout: the timeout (in seconds) when attempting to connect to the endpoint. This value is used for each connection attempt.

- chunkSize: the requested maximum size for log transfer packets that is used when the endpoint is contacted.

- idleMinWaitTime: the minimum wait time (in seconds) that the applier will intentionally idle before fetching more log data from the master in case the master has already sent all its log data. This wait time can be used to control the frequency with which the replication applier sends HTTP log fetch requests to the master in case there is no write activity on the master. This value will be ignored if set to 0.

- requestTimeout: the timeout (in seconds) for individual requests to the endpoint.

- database: the database name on the master (if not specified, defaults to the name of the local current database).

- requireFromPresent: if set to true, then the replication applier will check at start of its continuous replication if the start tick from the dump phase is still present on the master. If not, then there would be data loss. If requireFromPresent is true, the replication applier will abort with an appropriate error message. If set to false, then the replication applier will still start, and ignore the data loss.

- restrictType: an optional string value for collection filtering. When specified, the allowed values are include or exclude.

- password: the password to use when connecting to the master.

- idleMaxWaitTime: the maximum wait time (in seconds) that the applier will intentionally idle before fetching more log data from the master in case the master has already sent all its log data and there have been previous log fetch attempts that resulted in no more log data. This wait time can be used to control the maximum frequency with which the replication applier sends HTTP log fetch requests to the master in case there is no write activity on the master for longer periods. This configuration value will only be used if the option adaptivePolling is set to true. This value will be ignored if set to 0.

- adaptivePolling: whether or not the replication applier will use adaptive polling.

- restrictCollections: an optional array of collections for use with restrictType. If restrictType is include, only the specified collections will be sychronised. If restrictType is exclude, all but the specified collections will be synchronized. of type string

- maxConnectRetries: the maximum number of connection attempts the applier will make in a row. If the applier cannot establish a connection to the endpoint in this number of attempts, it will stop itself.

- initialSyncMaxWaitTime: the maximum wait time (in seconds) that the initial synchronization will wait for a response from the master when fetching initial collection data. This wait time can be used to control after what time the initial synchronization will give up waiting for a response and fail. This value is relevant even for continuous replication when autoResync is set to true because this may re-start the initial synchronization when the master cannot provide log data the slave requires. This value will be ignored if set to 0.

- connectionRetryWaitTime: the time (in seconds) that the applier will intentionally idle before it retries connecting to the master in case of connection problems. This value will be ignored if set to 0.

- autoResyncRetries: number of resynchronization retries that will be performed in a row when automatic resynchronization is enabled and kicks in. Setting this to 0 will effectively disable autoResync. Setting it to some other value will limit the number of retries that are performed. This helps preventing endless retries in case resynchronizations always fail.

- autoResync: whether or not the slave should perform an automatic resynchronization with the master in case the master cannot serve log data requested by the slave, or when the replication is started and no tick value can be found.

Starts a full data synchronization from a remote endpoint into the local ArangoDB database and afterwards starts the continuous replication. The operation works on a per-database level.

All local database data will be removed prior to the synchronization.

In case of success, the body of the response is a JSON object with the following attributes:

- state: a JSON object with the following sub-attributes:

- running: whether or not the applier is active and running

- lastAppliedContinuousTick: the last tick value from the continuous replication log the applier has applied.

- lastProcessedContinuousTick: the last tick value from the continuous replication log the applier has processed.

Regularly, the last applied and last processed tick values should be identical. For transactional operations, the replication applier will first process incoming log events before applying them, so the processed tick value might be higher than the applied tick value. This will be the case until the applier encounters the transaction commit log event for the transaction.

- lastAvailableContinuousTick: the last tick value the logger server can provide.

- time: the time on the applier server.

- totalRequests: the total number of requests the applier has made to the endpoint.

- totalFailedConnects: the total number of failed connection attempts the applier has made.

- totalEvents: the total number of log events the applier has processed.

- totalOperationsExcluded: the total number of log events excluded because of restrictCollections.

- progress: a JSON object with details about the replication applier progress. It contains the following sub-attributes if there is progress to report:

- message: a textual description of the progress

- time: the date and time the progress was logged

- failedConnects: the current number of failed connection attempts

- lastError: a JSON object with details about the last error that happened on the applier. It contains the following sub-attributes if there was an error:

- errorNum: a numerical error code

- errorMessage: a textual error description

- time: the date and time the error occurred

In case no error has occurred, lastError will be empty.

- server: a JSON object with the following sub-attributes:

- version: the applier server's version

- serverId: the applier server's id

- endpoint: the endpoint the applier is connected to (if applier is active) or will connect to (if applier is currently inactive)

- database: the name of the database the applier is connected to (if applier is active) or will connect to (if applier is currently inactive)

WARNING: calling this method will sychronize data from the collections found on the remote master to the local ArangoDB database. All data in the local collections will be purged and replaced with data from the master.

Use with caution!

Please also keep in mind that this command may take a long time to complete and return. This is because it will first do a full data synchronization with the master, which will take time roughly proportional to the amount of data.

Note: this method is not supported on a coordinator in a cluster.

Return Codes

- 200: is returned if the request was executed successfully.

- 400: is returned if the configuration is incomplete or malformed.

- 405: is returned when an invalid HTTP method is used.

- 500: is returned if an error occurred during sychronization or when starting the continuous replication.

- 501: is returned when this operation is called on a coordinator in a cluster.

Other Replication Commands

Return server id

fetch this servers uniq identifier

GET /_api/replication/server-id

Returns the servers id. The id is also returned by other replication API methods, and this method is an easy means of determining a server's id.

The body of the response is a JSON object with the attribute serverId. The server id is returned as a string.

Return Codes

- 200: is returned if the request was executed successfully.

- 405: is returned when an invalid HTTP method is used.

- 500: is returned if an error occurred while assembling the response.

Examples

shell> curl --dump - http://localhost:8529/_api/replication/server-id

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

{

"serverId" : "210709735347561"

}

HTTP tasks Interface

Following you have ArangoDB's HTTP Interface for Tasks.

There are also some examples provided for every API action.

Fetch all tasks or one task

Retrieves all currently active server tasks

GET /_api/tasks/

fetches all existing tasks on the server

- 200: The list of tasks

Examples

Fetching all tasks

shell> curl --dump - http://localhost:8529/_api/tasks

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

[

{

"id" : "15462632",

"name" : "user-defined task",

"type" : "periodic",

"period" : 1,

"created" : 1463943935.740431,

"command" : "(function () {

require('org/arangodb/foxx/queues/manager').manage();

})(params)",

"database" : "_system"

},

{

"id" : "statistics-gc",

"name" : "statistics-gc",

"type" : "periodic",

"period" : 450,

"created" : 1463943935.690809,

"command" : "require('org/arangodb/statistics').garbageCollector();",

"database" : "_system"

},

{

"id" : "statistics-average-collector",

"name" : "statistics-average-collector",

"type" : "periodic",

"period" : 900,

"created" : 1463943935.690336,

"command" : "require('org/arangodb/statistics').historianAverage();",

"database" : "_system"

},

{

"id" : "statistics-collector",

"name" : "statistics-collector",

"type" : "periodic",

"period" : 10,

"created" : 1463943935.690227,

"command" : "require('org/arangodb/statistics').historian();",

"database" : "_system"

}

]

Fetch one task with id

Retrieves one currently active server task

GET /_api/tasks/{id}

- id (required): The id of the task to fetch.

fetches one existing tasks on the server specified by id

- 200: The requested task

Examples

Fetching a single task by its id

shell> curl --dump - http://localhost:8529/_api/tasks/statistics-average-collector

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

trying to fetch a non-existing task

shell> curl --dump - http://localhost:8529/_api/tasks/non-existing-task

HTTP/1.1 404 Not Found

content-type: application/json; charset=utf-8

show response body

creates a task

creates a new task

POST /_api/tasks

Body Parameters

- period: number of seconds between the executions

- params: The parameters to be passed into command

- command: The JavaScript code to be executed

- name: The name of the task

- offset: Number of seconds initial delay

creates a new task with a generated id

Return Codes

- 400: If the post body is not accurate, a HTTP 400 is returned.

Examples

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/tasks/ <<EOF

{

"name" : "SampleTask",

"command" : "(function(params) { require('internal').print(params); })(params)",

"params" : {

"foo" : "bar",

"bar" : "foo"

},

"period" : 2

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

creates a task with id

registers a new task with a pre-defined id

PUT /_api/tasks/{id}

- id (required): The id of the task to create

Body Parameters

- period: number of seconds between the executions

- params: The parameters to be passed into command

- command: The JavaScript code to be executed

- name: The name of the task

- offset: Number of seconds initial delay

registers a new task with the specified id

Return Codes

- 400: If the task id already exists or the rest body is not accurate, HTTP 400 is returned.

Examples

shell> curl -X PUT --data-binary @- --dump - http://localhost:8529/_api/tasks/sampleTask <<EOF

{

"id" : "SampleTask",

"name" : "SampleTask",

"command" : "(function(params) { require('internal').print(params); })(params)",

"params" : {

"foo" : "bar",

"bar" : "foo"

},

"period" : 2

}

EOF

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

deletes the task with id

deletes one currently active server task

DELETE /_api/tasks/{id}

- id (required): The id of the task to delete.

Deletes the task identified by id on the server.

Return Codes

- 404: If the task id is unknown, then an HTTP 404 is returned.

Examples

trying to delete non existing task

shell> curl -X DELETE --dump - http://localhost:8529/_api/tasks/NoTaskWithThatName

HTTP/1.1 404 Not Found

content-type: application/json; charset=utf-8

show response body

Remove existing Task

shell> curl -X DELETE --dump - http://localhost:8529/_api/tasks/SampleTask

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

{

"error" : false,

"code" : 200

}

HTTP Interface for Administration and Monitoring

This is an introduction to ArangoDB's HTTP interface for administration and monitoring of the server.

Read global log from the server

returns the log files

GET /_admin/log

Query Parameters

- upto (optional): Returns all log entries up to log level upto. Note that upto must be:

- fatal or 0

- error or 1

- warning or 2

- info or 3

- debug or 4 The default value is info.

- level (optional): Returns all log entries of log level level. Note that the query parameters upto and level are mutually exclusive.

- start (optional): Returns all log entries such that their log entry identifier (lid value) is greater or equal to start.

- size (optional): Restricts the result to at most size log entries.

- offset (optional): Starts to return log entries skipping the first offset log entries. offset and size can be used for pagination.

- search (optional): Only return the log entries containing the text specified in search.

- sort (optional): Sort the log entries either ascending (if sort is asc) or descending (if sort is desc) according to their lid values. Note that the lid imposes a chronological order. The default value is asc.

Returns fatal, error, warning or info log messages from the server's global log. The result is a JSON object with the following attributes:

- lid: a list of log entry identifiers. Each log message is uniquely identified by its @LIT{lid} and the identifiers are in ascending order.

- level: a list of the log-levels for all log entries.

- timestamp: a list of the timestamps as seconds since 1970-01-01 for all log entries.

- text a list of the texts of all log entries

- totalAmount: the total amount of log entries before pagination.

Return Codes

- 400: is returned if invalid values are specified for upto or level.

- 403: is returned if the log is requested for any database other than _system.

- 500: is returned if the server cannot generate the result due to an out-of-memory error.

Reloads the routing information

Reload the routing table.

POST /_admin/routing/reload

Reloads the routing information from the collection routing.

Return Codes

- 200: Routing information was reloaded successfully.

Read the statistics

return the statistics information

GET /_admin/statistics

Returns the statistics information. The returned object contains the statistics figures grouped together according to the description returned by _admin/statistics-description. For instance, to access a figure userTime from the group system, you first select the sub-object describing the group stored in system and in that sub-object the value for userTime is stored in the attribute of the same name.

In case of a distribution, the returned object contains the total count in count and the distribution list in counts. The sum (or total) of the individual values is returned in sum.

Return Codes

- 200: Statistics were returned successfully.

Examples

shell> curl --dump - http://localhost:8529/_admin/statistics

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Statistics description

fetch descriptive info of statistics

GET /_admin/statistics-description

Returns a description of the statistics returned by /_admin/statistics. The returned objects contains an array of statistics groups in the attribute groups and an array of statistics figures in the attribute figures.

A statistics group is described by

- group: The identifier of the group.

- name: The name of the group.

- description: A description of the group.

A statistics figure is described by

- group: The identifier of the group to which this figure belongs.

- identifier: The identifier of the figure. It is unique within the group.

- name: The name of the figure.

- description: A description of the figure.

- type: Either current, accumulated, or distribution.

- cuts: The distribution vector.

- units: Units in which the figure is measured.

Return Codes

- 200: Description was returned successfully.

Examples

shell> curl --dump - http://localhost:8529/_admin/statistics-description

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Return role of a server in a cluster

Get to know whether this server is a Coordinator or DB-Server

GET /_admin/server/role

Returns the role of a server in a cluster. The role is returned in the role attribute of the result. Possible return values for role are:

- COORDINATOR: the server is a coordinator in a cluster

- PRIMARY: the server is a primary database server in a cluster

- SECONDARY: the server is a secondary database server in a cluster

- UNDEFINED: in a cluster, UNDEFINED is returned if the server role cannot be determined. On a single server, UNDEFINED is the only possible return value.

Return Codes

- 200: Is returned in all cases.

Queries statistics of DBserver

allows to query the statistics of a DBserver in the cluster

GET /_admin/clusterStatistics

Query Parameters

- DBserver (required):

Queries the statistics of the given DBserver

Return Codes

- 200: is returned when everything went well.

- 400: the parameter DBserver was not given or is not the ID of a DBserver

- 403: server is not a coordinator.

HTTP Interface for User Management

This is an introduction to ArangoDB's HTTP interface for managing users.

The interface provides a simple means to add, update, and remove users. All users managed through this interface will be stored in the system collection _users.

This specialized interface intentionally does not provide all functionality that is available in the regular document REST API.

You should not manipulate the _users collection directly. While the user management interface aims to maintain backwards compatibility, the underlying collection is now managed by the bundled users Foxx app.

Please note that user operations are not included in ArangoDB's replication.

Create User

Create a new user.

POST /_api/user

The following data need to be passed in a JSON representation in the body of the POST request:

- user: The name of the user as a string. This is mandatory.

- passwd: The user password as a string. If no password is specified, the empty string will be used. If you pass the special value ARANGODB_DEFAULT_ROOT_PASSWORD, the password will be set the value stored in the environment variable  ARANGODB_DEFAULT_ROOT_PASSWORD . This can be used to pass an instance variable into ArangoDB. For example, the instance identifier from Amazon.

- active: An optional flag that specifies whether the user is active. If not specified, this will default to true

- extra: An optional JSON object with arbitrary extra data about the user

- changePassword: An optional flag that specifies whethers the user must change the password or not. If not specified, this will default to false. If set to true, the only operations allowed are PUT /_api/user or PATCH /_api/user. All other operations executed by the user will result in an HTTP 403.

If the user can be added by the server, the server will respond with HTTP 201. In case of success, the returned JSON object has the following properties:

- error: Boolean flag to indicate that an error occurred (false in this case)

- code: The HTTP status code

In case of error, the body of the response will contain a JSON object with additional error details. The object has the following attributes:

- error: Boolean flag to indicate that an error occurred (true in this case)

- code: The HTTP status code

- errorNum: The server error number

- errorMessage: A descriptive error message

Return Codes

- 201: Returned if the user can be added by the server

- 400: If the JSON representation is malformed or mandatory data is missing from the request.

Replace User

replace an existing user with a new one.

PUT /_api/user/{user}

Path Parameters

- user (required): The name of the user

Replaces the data of an existing user. The name of an existing user must be specified in user.

The following data can to be passed in a JSON representation in the body of the POST request:

- passwd: The user password as a string. Specifying a password is mandatory, but the empty string is allowed for passwords

- active: An optional flag that specifies whether the user is active. If not specified, this will default to true

- extra: An optional JSON object with arbitrary extra data about the user

- changePassword: An optional flag that specifies whether the user must change the password or not. If not specified, this will default to false

If the user can be replaced by the server, the server will respond with HTTP 200.

In case of success, the returned JSON object has the following properties:

- error: Boolean flag to indicate that an error occurred (false in this case)

- code: The HTTP status code

In case of error, the body of the response will contain a JSON object with additional error details. The object has the following attributes:

- error: Boolean flag to indicate that an error occurred (true in this case)

- code: The HTTP status code

- errorNum: The server error number

- errorMessage: A descriptive error message

Return Codes

- 200: Is returned if the user data can be replaced by the server

- 400: The JSON representation is malformed or mandatory data is missing from the request

- 404: The specified user does not exist

Update User

modify attributes of an existing user

PATCH /_api/user/{user}

Path Parameters

- user (required): The name of the user

Partially updates the data of an existing user. The name of an existing user must be specified in user.

The following data can be passed in a JSON representation in the body of the POST request:

- passwd: The user password as a string. Specifying a password is optional. If not specified, the previously existing value will not be modified.

- active: An optional flag that specifies whether the user is active. If not specified, the previously existing value will not be modified.

- extra: An optional JSON object with arbitrary extra data about the user. If not specified, the previously existing value will not be modified.

- changePassword: An optional flag that specifies whether the user must change the password or not. If not specified, the previously existing value will not be modified.

If the user can be updated by the server, the server will respond with HTTP 200.

In case of success, the returned JSON object has the following properties:

- error: Boolean flag to indicate that an error occurred (false in this case)

- code: The HTTP status code

In case of error, the body of the response will contain a JSON object with additional error details. The object has the following attributes:

- error: Boolean flag to indicate that an error occurred (true in this case)

- code: The HTTP status code

- errorNum: The server error number

- errorMessage: A descriptive error message

Return Codes

- 200: Is returned if the user data can be replaced by the server

- 400: The JSON representation is malformed or mandatory data is missing from the request

- 404: The specified user does not exist

Remove User

delete a user permanently.

DELETE /_api/user/{user}

Path Parameters

- user (required): The name of the user

Removes an existing user, identified by user.

If the user can be removed, the server will respond with HTTP 202. In case of success, the returned JSON object has the following properties:

- error: Boolean flag to indicate that an error occurred (false in this case)

- code: The HTTP status code

In case of error, the body of the response will contain a JSON object with additional error details. The object has the following attributes:

- error: Boolean flag to indicate that an error occurred (true in this case)

- code: The HTTP status code

- errorNum: The server error number

- errorMessage: A descriptive error message

Return Codes

- 202: Is returned if the user was removed by the server

- 404: The specified user does not exist

Fetch User

fetch the properties of a user.

GET /_api/user/{user}

Path Parameters

- user (required): The name of the user

Fetches data about the specified user.

The call will return a JSON object with at least the following attributes on success:

- user: The name of the user as a string.

- active: An optional flag that specifies whether the user is active.

- extra: An optional JSON object with arbitrary extra data about the user.

- changePassword: An optional flag that specifies whether the user must change the password or not.

Return Codes

- 200: The user was found

- 404: The user with the specified name does not exist

List available Users

fetch the properties of a user.

GET /_api/user/

Fetches data about all users.

The call will return a JSON object with at least the following attributes on success:

- user: The name of the user as a string.

- active: An optional flag that specifies whether the user is active.

- extra: An optional JSON object with arbitrary extra data about the user.

- changePassword: An optional flag that specifies whether the user must change the password or not.

Return Codes

- 200: The users that were found

HTTP Interface for Async Results Management

Request Execution

ArangoDB provides various methods of executing client requests. Clients can choose the appropriate method on a per-request level based on their throughput, control flow, and durability requirements.

Blocking execution

ArangoDB is a multi-threaded server, allowing the processing of multiple client requests at the same time. Communication handling and the actual work can be performed by multiple worker threads in parallel.

Though multiple clients can connect and send their requests in parallel to ArangoDB, clients may need to wait for their requests to be processed.

By default, the server will fully process an incoming request and then return the result to the client. The client must wait for the server's response before it can send additional requests over the connection. For clients that are single-threaded or not event-driven, waiting for the full server response may be non-optimal.

Furthermore, please note that even if the client closes the HTTP connection, the request running on the server will still continue until it is complete and only then notice that the client no longer listens. Thus closing the connection does not help to abort a long running query! See below under Async Execution and later Result Retrieval and HttpJobPutCancel for details.

Fire and Forget

To mitigate client blocking issues, ArangoDB since version 1.4. offers a generic mechanism for non-blocking requests: if clients add the HTTP header x-arango-async: true to their requests, ArangoDB will put the request into an in-memory task queue and return an HTTP 202 (accepted) response to the client instantly. The server will execute the tasks from the queue asynchronously, decoupling the client requests and the actual work.

This allows for much higher throughput than if clients would wait for the server's response. The downside is that the response that is sent to the client is always the same (a generic HTTP 202) and clients cannot make a decision based on the actual operation's result. In fact, the operation might have not even been executed at the time the generic response has reached the client. Clients can thus not rely on their requests having been processed successfully.

The asynchronous task queue on the server is not persisted, meaning not-yet processed tasks from the queue might be lost in case of a crash.

Clients should thus not send the extra header when they have strict durability requirements or if they rely on result of the sent operation for further actions.

The maximum number of queued tasks is determined by the startup option -scheduler.maximal-queue-size. If more than this number of tasks are already queued, the server will reject the request with an HTTP 500 error.

Finally, please note that it is not possible to cancel such a non-blocking request after the fact. If you need to cancel requests, use Async Execution and later Result Retrieval and HttpJobPutCancel below.

Async Execution and later Result Retrieval

By adding the HTTP header x-arango-async: store to a request, clients can instruct the ArangoDB server to execute the operation asynchronously as above "above", but also store the operation result in memory for a later retrieval. The server will return a job id in the HTTP response header x-arango-async-id. The client can use this id in conjunction with the HTTP API at /_api/job, which is described in detail in this manual.

Clients can ask the ArangoDB server via the async jobs API which results are ready for retrieval, and which are not. Clients can also use the async jobs API to retrieve the original results of an already executed async job by passing it the originally returned job id. The server will then return the job result as if the job was executed normally. Furthermore, clients can cancel running async jobs by their job id, see HttpJobPutCancel.

ArangoDB will keep all results of jobs initiated with the x-arango-async: store header. Results are removed from the server only if a client explicitly asks the server for a specific result.

The async jobs API also provides methods for garbage collection that clients can use to get rid of "old" not fetched results. Clients should call this method periodically because ArangoDB does not artificially limit the number of not-yet-fetched results.

It is thus a client responsibility to store only as many results as needed and to fetch available results as soon as possible, or at least to clean up not fetched results from time to time.

The job queue and the results are kept in memory only on the server, so they might be lost in case of a crash.

Canceling asynchronous jobs

As mentioned above it is possible to cancel an asynchronously running job using its job ID. This is done with a PUT request as described in HttpJobPutCancel.

However, a few words of explanation about what happens behind the scenes are in order. Firstly, a running async query can internally be executed by C++ code or by JavaScript code. For example CRUD operations are executed directly in C++, whereas AQL queries and transactions are executed by JavaScript code. The job cancelation only works for JavaScript code, since the mechanism used is simply to trigger an uncatchable exception in the JavaScript thread, which will be caught on the C++ level, which in turn leads to the cancelation of the job. No result can be retrieved later, since all data about the request is discarded.

If you cancel a job running on a coordinator of a cluster (Sharding), then only the code running on the coordinator is stopped, there may remain tasks within the cluster which have already been distributed to the DBservers and it is currently not possible to cancel them as well.

Async Execution and Authentication

If a request requires authentication, the authentication procedure is run before queueing. The request will only be queued if it valid credentials and the authentication succeeds. If the request does not contain valid credentials, it will not be queued but rejected instantly in the same way as a "regular", non-queued request.

Managing Async Results via HTTP

Return result of an async job

fetches a job result and removes it from the queue

PUT /_api/job/{job-id}

Path Parameters

- job-id (required): The async job id.

Returns the result of an async job identified by job-id. If the async job result is present on the server, the result will be removed from the list of result. That means this method can be called for each job-id once. The method will return the original job result's headers and body, plus the additional HTTP header x-arango-async-job-id. If this header is present, then the job was found and the response contains the original job's result. If the header is not present, the job was not found and the response contains status information from the job manager.

Return Codes

- 204: is returned if the job requested via job-id is still in the queue of pending (or not yet finished) jobs. In this case, no x-arango-async-id HTTP header will be returned.

- 400: is returned if no job-id was specified in the request. In this case, no x-arango-async-id HTTP header will be returned.

- 404: is returned if the job was not found or already deleted or fetched from the job result list. In this case, no x-arango-async-id HTTP header will be returned.

Examples

Not providing a job-id:

shell> curl -X PUT --dump - http://localhost:8529/_api/job

HTTP/1.1 400 Bad Request

content-type: application/json; charset=utf-8

show response body

Providing a job-id for a non-existing job:

shell> curl -X PUT --dump - http://localhost:8529/_api/job/notthere

HTTP/1.1 404 Not Found

content-type: application/json; charset=utf-8

show response body

Fetching the result of an HTTP GET job:

shell> curl -X PUT --header 'x-arango-async: store' --dump - http://localhost:8529/_api/version

HTTP/1.1 202 Accepted

content-type: text/plain; charset=utf-8

x-arango-async-id: 626716904

shell> curl -X PUT --dump - http://localhost:8529/_api/job/626716904

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

x-arango-async-id: 626716904

{

"server" : "arango",

"version" : "2.8.9"

}

Fetching the result of an HTTP POST job that failed:

shell> curl -X PUT --header 'x-arango-async: store' --data-binary @- --dump - http://localhost:8529/_api/collection <<EOF

{

"name" : " this name is invalid "

}

EOF

HTTP/1.1 202 Accepted

content-type: text/plain; charset=utf-8

x-arango-async-id: 626782440

shell> curl -X PUT --dump - http://localhost:8529/_api/job/626782440

HTTP/1.1 400 Bad Request

content-type: application/json; charset=utf-8

x-arango-async-id: 626782440

show response body

Cancel async job

cancels an async job

PUT /_api/job/{job-id}/cancel

Path Parameters

- job-id (required): The async job id.

Cancels the currently running job identified by job-id. Note that it still might take some time to actually cancel the running async job.

Return Codes

- 200: cancel has been initiated.

- 400: is returned if no job-id was specified in the request. In this case, no x-arango-async-id HTTP header will be returned.

- 404: is returned if the job was not found or already deleted or fetched from the job result list. In this case, no x-arango-async-id HTTP header will be returned.

Examples

shell> curl -X POST --header 'x-arango-async: store' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF

{

"query" : "FOR i IN 1..10 FOR j IN 1..10 LET x = sleep(1.0) FILTER i == 5 && j == 5 RETURN 42"

}

EOF

HTTP/1.1 202 Accepted

content-type: text/plain; charset=utf-8

x-arango-async-id: 626389224

shell> curl --dump - http://localhost:8529/_api/job/pending

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

[

"626389224"

]

shell> curl -X PUT --dump - http://localhost:8529/_api/job/626389224/cancel

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Deletes async job

deletes an async job result

DELETE /_api/job/{type}

Path Parameters

- type (required): The type of jobs to delete. type can be:

- all: Deletes all jobs results. Currently executing or queued async jobs will not be stopped by this call.

- expired: Deletes expired results. To determine the expiration status of a result, pass the stamp query parameter. stamp needs to be a UNIX timestamp, and all async job results created at a lower timestamp will be deleted.

- an actual job-id: In this case, the call will remove the result of the specified async job. If the job is currently executing or queued, it will not be aborted.

Query Parameters

- stamp (optional):

A UNIX timestamp specifying the expiration threshold when type is expired.

Deletes either all job results, expired job results, or the result of a specific job. Clients can use this method to perform an eventual garbage collection of job results.

Return Codes

- 200: is returned if the deletion operation was carried out successfully. This code will also be returned if no results were deleted.

- 400: is returned if type is not specified or has an invalid value.

- 404: is returned if type is a job-id but no async job with the specified id was found.

Examples

Deleting all jobs:

shell> curl -X PUT --header 'x-arango-async: store' --dump - http://localhost:8529/_api/version

HTTP/1.1 202 Accepted

content-type: text/plain; charset=utf-8

x-arango-async-id: 626520296

shell> curl -X DELETE --dump - http://localhost:8529/_api/job/all

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

{

"result" : true

}

Deleting expired jobs:

shell> curl -X PUT --header 'x-arango-async: store' --dump - http://localhost:8529/_api/version

HTTP/1.1 202 Accepted

content-type: text/plain; charset=utf-8

x-arango-async-id: 626585832

shell> curl --dump - http://localhost:8529/_admin/time

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Deleting the result of a specific job:

shell> curl -X PUT --header 'x-arango-async: store' --dump - http://localhost:8529/_api/version

HTTP/1.1 202 Accepted

content-type: text/plain; charset=utf-8

x-arango-async-id: 626651368

shell> curl -X DELETE --dump - http://localhost:8529/_api/job/626651368

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

{

"result" : true

}

Deleting the result of a non-existing job:

shell> curl -X DELETE --dump - http://localhost:8529/_api/job/AreYouThere

HTTP/1.1 404 Not Found

content-type: application/json; charset=utf-8

show response body

Returns async job

Returns the status of a specific job

GET /_api/job/{job-id}

Path Parameters

- job-id (required): The async job id.

Returns the processing status of the specified job. The processing status can be determined by peeking into the HTTP response code of the response.

Return Codes

- 200: is returned if the job requested via job-id has been executed and its result is ready to fetch.

- 204: is returned if the job requested via job-id is still in the queue of pending (or not yet finished) jobs.

- 404: is returned if the job was not found or already deleted or fetched from the job result list.

Examples

Querying the status of a done job:

shell> curl -X PUT --header 'x-arango-async: store' --dump - http://localhost:8529/_api/version

HTTP/1.1 202 Accepted

content-type: text/plain; charset=utf-8

x-arango-async-id: 627044584

shell> curl -X PUT --dump - http://localhost:8529/_api/job/627044584

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

x-arango-async-id: 627044584

{

"server" : "arango",

"version" : "2.8.9"

}

Querying the status of a pending job: (we create a sleep job therefore...)

shell> curl --header 'x-arango-async: store' --dump - http://localhost:8529/_admin/sleep?duration=30

HTTP/1.1 202 Accepted

content-type: text/plain; charset=utf-8

x-arango-async-id: 627110120

shell> curl --dump - http://localhost:8529/_api/job/627110120

HTTP/1.1 204 No Content

content-type: text/plain; charset=utf-8

Returns list of async jobs

Returns the ids of job results with a specific status

GET /_api/job/{type}

Path Parameters

- type (required): The type of jobs to return. The type can be either done or pending. Setting the type to done will make the method return the ids of already completed async jobs for which results can be fetched. Setting the type to pending will return the ids of not yet finished async jobs.

Query Parameters

- count (optional):

The maximum number of ids to return per call. If not specified, a server-defined maximum value will be used.

Returns the list of ids of async jobs with a specific status (either done or pending). The list can be used by the client to get an overview of the job system status and to retrieve completed job results later.

Return Codes

- 200: is returned if the list can be compiled successfully. Note: the list might be empty.

- 400: is returned if type is not specified or has an invalid value.

Examples

Fetching the list of done jobs:

shell> curl -X PUT --header 'x-arango-async: store' --dump - http://localhost:8529/_api/version

HTTP/1.1 202 Accepted

content-type: text/plain; charset=utf-8

x-arango-async-id: 626847976

shell> curl --dump - http://localhost:8529/_api/job/done

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

[

"626847976"

]

Fetching the list of pending jobs:

shell> curl -X PUT --header 'x-arango-async: store' --dump - http://localhost:8529/_api/version

HTTP/1.1 202 Accepted

content-type: text/plain; charset=utf-8

x-arango-async-id: 626913512

shell> curl --dump - http://localhost:8529/_api/job/pending

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

[ ]

Querying the status of a pending job: (we create a sleep job therefore...)

shell> curl --header 'x-arango-async: store' --dump - http://localhost:8529/_admin/sleep?duration=30

HTTP/1.1 202 Accepted

content-type: text/plain; charset=utf-8

x-arango-async-id: 626979048

shell> curl --dump - http://localhost:8529/_api/job/pending

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

[

"626979048"

]

shell> curl -X DELETE --dump - http://localhost:8529/_api/job/626979048

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

{

"result" : true

}

HTTP Interface for Endpoints

The ArangoDB server can listen for incoming requests on multiple endpoints.

The endpoints are normally specified either in ArangoDB's configuration file or on the command-line, using the "--server.endpoint" option. The default endpoint for ArangoDB is tcp://127.0.0.1:8529 or tcp://localhost:8529.

The number of endpoints can also be changed at runtime using the API described below. Each endpoint can optionally be restricted to a specific list of databases only, thus allowing the usage of different port numbers for different databases.

This may be useful in multi-tenant setups. A multi-endpoint setup may also be useful to turn on encrypted communication for just specific databases.

The HTTP interface provides operations to add new endpoints at runtime, and optionally restrict them for use with specific databases. The interface also can be used to update existing endpoints or remove them at runtime.

Please note that all endpoint management operations can only be accessed via the default database (_system) and none of the other databases.

Managing Endpoints via HTTP

Return list of all endpoints

returns a list of all endpoints

GET /_api/endpoint

Returns an array of all configured endpoints the server is listening on. For each endpoint, the array of allowed databases is returned too if set.

The result is a JSON object which has the endpoints as keys, and an array of mapped database names as values for each endpoint.

If an array of mapped databases is empty, it means that all databases can be accessed via the endpoint. If an array of mapped databases contains more than one database name, this means that any of the databases might be accessed via the endpoint, and the first database in the arry will be treated as the default database for the endpoint. The default database will be used when an incoming request does not specify a database name in the request explicitly.

Note: retrieving the array of all endpoints is allowed in the system database only. Calling this action in any other database will make the server return an error.

Return Codes

- 200: is returned when the array of endpoints can be determined successfully.

- 400: is returned if the action is not carried out in the system database.

- 405: The server will respond with HTTP 405 if an unsupported HTTP method is used.

Examples

shell> curl --dump - http://localhost:8529/_api/endpoint

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

[

{

"endpoint" : "tcp://127.0.0.1:36839",

"databases" : [ ]

}

]

HTTP Interface for Sharding

Sharding only should be used by developers!

Execute cluster roundtrip

executes a cluster roundtrip for sharding

GET /_admin/cluster-test

Executes a cluster roundtrip from a coordinator to a DB server and back. This call only works in a coordinator node in a cluster. One can and should append an arbitrary path to the URL and the part after /_admin/cluster-test is used as the path of the HTTP request which is sent from the coordinator to a DB node. Likewise, any form data appended to the URL is forwarded in the request to the DB node. This handler takes care of all request types (see below) and uses the same request type in its request to the DB node.

The following HTTP headers are interpreted in a special way:

- X-Shard-ID: This specifies the ID of the shard to which the cluster request is sent and thus tells the system to which DB server to send the cluster request. Note that the mapping from the shard ID to the responsible server has to be defined in the agency under Current/ShardLocation/. One has to give this header, otherwise the system does not know where to send the request.

- X-Client-Transaction-ID: the value of this header is taken as the client transaction ID for the request

- X-Timeout: specifies a timeout in seconds for the cluster operation. If the answer does not arrive within the specified timeout, an corresponding error is returned and any subsequent real answer is ignored. The default if not given is 24 hours.

- X-Synchronous-Mode: If set to true the test function uses synchronous mode, otherwise the default asynchronous operation mode is used. This is mainly for debugging purposes.

- Host: This header is ignored and not forwarded to the DB server.

- User-Agent: This header is ignored and not forwarded to the DB server.

All other HTTP headers and the body of the request (if present, see other HTTP methods below) are forwarded as given in the original request.

In asynchronous mode the DB server answers with an HTTP request of its own, in synchronous mode it sends a HTTP response. In both cases the headers and the body are used to produce the HTTP response of this API call.

Return Codes

The return code can be anything the cluster request returns, as well as:

- 200: is returned when everything went well, or if a timeout occurred. In the latter case a body of type application/json indicating the timeout is returned.

- 403: is returned if ArangoDB is not running in cluster mode.

- 404: is returned if ArangoDB was not compiled for cluster operation.

Execute cluster roundtrip

executes a cluster roundtrip for sharding

POST /_admin/cluster-test

Post Body body (required): The body can be any type and is simply forwarded.

See GET method.

Execute cluster roundtrip

executes a cluster roundtrip for sharding

PUT /_admin/cluster-test

Post Body body (required):

See GET method. The body can be any type and is simply forwarded.

Delete cluster roundtrip

executes a cluster roundtrip for sharding

DELETE /_admin/cluster-test

See GET method.

Update cluster roundtrip

executes a cluster roundtrip for sharding

PATCH /_admin/cluster-test

Post Body body (required):

See GET method. The body can be any type and is simply forwarded.

Execute cluster roundtrip

executes a cluster roundtrip for sharding

HEAD /_admin/cluster-test

See GET method.

Produce cluster startup plan

exposes the cluster planning functionality

POST /_admin/clusterPlanner

Post Body clusterPlan (required): A cluster plan object

Given a description of a cluster, this plans the details of a cluster and returns a JSON description of a plan to start up this cluster.

Return Codes

- 200: is returned when everything went well.

- 400: the posted body was not valid JSON.

execute startup commands

exposes the dispatcher functionality to start up, shutdown, relaunch, upgrade or cleanup a cluster according to a cluster plan as for example provided by the kickstarter.

POST /_admin/clusterDispatch

Query Parameters

Body Parameters

- action: can be one of the following:

- "launch": the cluster is launched for the first time, all data directories and log files are cleaned and created

- "shutdown": the cluster is shut down, the additional property runInfo (see below) must be bound as well

- "relaunch": the cluster is launched again, all data directories and log files are untouched and need to be there already

- "cleanup": use this after a shutdown to remove all data in the data directories and all log files, use with caution

- "isHealthy": checks whether or not the processes involved in the cluster are running or not. The additional property runInfo (see above) must be bound as well

- "upgrade": performs an upgrade of a cluster, to this end, the agency is started, and then every server is once started with the "--upgrade" option, and then normally. Finally, the script "verion-check.js" is run on one of the coordinators for the cluster.

- clusterPlan: is a cluster plan (see JSF_cluster_planner_POST),

- runInfo: this is needed for the "shutdown" and "isHealthy" actions only and should be the structure that "launch", "relaunch" or "upgrade" returned. It contains runtime information like process IDs.

- myname: is the ID of this dispatcher, this is used to decide which commands are executed locally and which are forwarded to other dispatchers

The body must be an object with the following properties:

This call executes the plan by either doing the work personally or by delegating to other dispatchers.

Return Codes

- 200: is returned when everything went well.

- 400: the posted body was not valid JSON, or something went wrong with the startup.

Check port

allows to check whether a given port is usable

GET /_admin/clusterCheckPort

Query Parameters

- port (required):

Checks whether the requested port is usable.

Return Codes

- 200: is returned when everything went well.

- 400: the parameter port was not given or is no integer.

HTTP Interface for Miscellaneous functions

This is an overview of ArangoDB's HTTP interface for miscellaneous functions.

Return server version

returns the server version number

GET /_api/version

Query Parameters

- details (optional): If set to true, the response will contain a details attribute with additional information about included components and their versions. The attribute names and internals of the details object may vary depending on platform and ArangoDB version.

Returns the server name and version number. The response is a JSON object with the following attributes:

Return Codes

- 200: is returned in all cases.

Reply Body

- version: the server version string. The string has the format "major.minor.sub". major and minor will be numeric, and sub may contain a number or a textual version.

- details: an optional JSON object with additional details. This is returned only if the details query parameter is set to true in the request.

- server: will always contain arango

Examples

Return the version information

shell> curl --dump - http://localhost:8529/_api/version

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

{

"server" : "arango",

"version" : "2.8.9"

}

Return the version information with details

shell> curl --dump - http://localhost:8529/_api/version?details=true

HTTP/1.1 200 OK

content-type: application/json; charset=utf-8

show response body

Flushes the write-ahead log

Sync the WAL to disk.

PUT /_admin/wal/flush

Path Parameters

- waitForSync (optional): Whether or not the operation should block until the not-yet synchronized data in the write-ahead log was synchronized to disk.

- waitForCollector (optional): Whether or not the operation should block until the data in the flushed log has been collected by the write-ahead log garbage collector. Note that setting this option to true might block for a long time if there are long-running transactions and the write-ahead log garbage collector cannot finish garbage collection.

Flushes the write-ahead log. By flushing the currently active write-ahead logfile, the data in it can be transferred to collection journals and datafiles. This is useful to ensure that all data for a collection is present in the collection journals and datafiles, for example, when dumping the data of a collection.

Return Codes

- 200: Is returned if the operation succeeds.

- 405: is returned when an invalid HTTP method is used.

Retrieves the configuration of the write-ahead log

fetch the current configuration.

GET /_admin/wal/properties

Retrieves the configuration of the write-ahead log. The result is a JSON object with the following attributes:

- allowOversizeEntries: whether or not operations that are bigger than a single logfile can be executed and stored

- logfileSize: the size of each write-ahead logfile

- historicLogfiles: the maximum number of historic logfiles to keep

- reserveLogfiles: the maximum number of reserve logfiles that ArangoDB allocates in the background

- syncInterval: the interval for automatic synchronization of not-yet synchronized write-ahead log data (in milliseconds)

- throttleWait: the maximum wait time that operations will wait before they get aborted if case of write-throttling (in milliseconds)

- throttleWhenPending: the number of unprocessed garbage-collection operations that, when reached, will activate write-throttling. A value of 0 means that write-throttling will not be triggered.

Return Codes

- 200: Is returned if the operation succeeds.

- 405: is returned when an invalid HTTP method is used.

Configures the write-ahead log

configure parameters of the wal

PUT /_admin/wal/properties

Configures the behavior of the write-ahead log. The body of the request must be a JSON object with the following attributes:

- allowOversizeEntries: whether or not operations that are bigger than a single logfile can be executed and stored

- logfileSize: the size of each write-ahead logfile

- historicLogfiles: the maximum number of historic logfiles to keep

- reserveLogfiles: the maximum number of reserve logfiles that ArangoDB allocates in the background

- throttleWait: the maximum wait time that operations will wait before they get aborted if case of write-throttling (in milliseconds)

- throttleWhenPending: the number of unprocessed garbage-collection operations that, when reached, will activate write-throttling. A value of 0 means that write-throttling will not be triggered.

Specifying any of the above attributes is optional. Not specified attributes will be ignored and the configuration for them will not be modified.

Return Codes

- 200: Is returned if the operation succeeds.

- 405: is returned when an invalid HTTP method is used.

Returns information about the currently running transactions

returns information about the currently running transactions

GET /_admin/wal/transactions

Returns information about the currently running transactions. The result is a JSON object with the following attributes:

- runningTransactions: number of currently running transactions

- minLastCollected: minimum id of the last collected logfile (at the start of each running transaction). This is null if no transaction is running.

- minLastSealed: minimum id of the last sealed logfile (at the start of each running transaction). This is null if no transaction is running.

Return Codes

- 200: Is returned if the operation succeeds.

- 405: is returned when an invalid HTTP method is used.

Return system time

Get the current time of the system

GET /_admin/time

The call returns an object with the attribute time. This contains the current system time as a Unix timestamp with microsecond precision.

Return Codes

- 200: Time was returned successfully.

Return current request

Send back what was sent in, headers, post body etc.

GET /_admin/echo

The call returns an object with the following attributes:

- headers: object with HTTP headers received

- requestType: the HTTP request method (e.g. GET)

- parameters: object with query parameters received

Return Codes

- 200: Echo was returned successfully.

Return current request and continues

Send back what was sent in, headers, post body etc.

GET /_admin/long_echo

The call returns an object with the following attributes:

- headers: object with HTTP headers received

- requestType: the HTTP request method (e.g. GET)

- parameters: object with query parameters received

Return Codes

- 200: Echo was returned successfully.

Return the required version of the database

returns the version of the database.

GET /_admin/database/target-version

Returns the database-version that this server requires. The version is returned in the version attribute of the result.

Return Codes

- 200: Is returned in all cases.

Sleep for a specified amount of seconds

Suspend the execution for a specified duration before returnig

GET /_admin/sleep

Query Parameters

- duration (required): wait  duration  seconds until the reply is sent.

The call returns an object with the attribute duration. This takes as many seconds as the duration argument says.

Return Codes

- 200: Sleep was conducted successfully.

Initiate shutdown sequence

initiates the shutdown sequence

GET /_admin/shutdown

This call initiates a clean shutdown sequence.

Return Codes

- 200: is returned in all cases.

Runs tests on server

show the available unittests on the server.

POST /_admin/test

Post Body body (required): A JSON object containing an attribute tests which lists the files containing the test suites.

Executes the specified tests on the server and returns an object with the test results. The object has an attribute "error" which states whether any error occurred. The object also has an attribute "passed" which indicates which tests passed and which did not.

Execute program

Execute a script on the server.

POST /_admin/execute

Post Body body (required): The body to be executed.

Executes the javascript code in the body on the server as the body of a function with no arguments. If you have a return statement then the return value you produce will be returned as content type application/json. If the parameter returnAsJSON is set to true, the result will be a JSON object describing the return value directly, otherwise a string produced by JSON.stringify will be returned.

HTTP Interface for Bulk Imports

ArangoDB provides an HTTP interface to import multiple documents at once into a collection. This is known as a bulk import.

The data uploaded must be provided in JSON format. There are two mechanisms to import the data:

- self-contained JSON documents: in this case, each document contains all attribute names and values. Attribute names may be completely different among the documents uploaded

- attribute names plus document data: in this case, the first array must contain the attribute names of the documents that follow. The following arrays containing only the attribute values. Attribute values will be mapped to the attribute names by positions.

The endpoint address is /_api/import for both input mechanisms. Data must be sent to this URL using an HTTP POST request. The data to import must be contained in the body of the POST request.

The collection query parameter must be used to specify the target collection for the import. The optional query parameter createCollection can be used to create a non-existing collection during the import. If not used, importing data into a non-existing collection will produce an error. Please note that the createCollection flag can only be used to create document collections, not edge collections.

The waitForSync query parameter can be set to true to make the import only return if all documents have been synced to disk.

The complete query parameter can be set to true to make the entire import fail if any of the uploaded documents is invalid and cannot be imported. In this case, no documents will be imported by the import run, even if a failure happens at the end of the import.

If complete has a value other than true, valid documents will be imported while invalid documents will be rejected, meaning only some of the uploaded documents might have been imported.

The details query parameter can be set to true to make the import API return details about documents that could not be imported. If details is true, then the result will also contain a details attribute which is an array of detailed error messages. If the details is set to false or omitted, no details will be returned.

imports document values

imports documents from JSON-encoded lists

POST /_api/import#document

Post Body documents (required): The body must consist of JSON-encoded arrays of attribute values, with one line per document. The first row of the request must be a JSON-encoded array of attribute names. These attribute names are used for the data in the subsequent lines.

Query Parameters

- collection (required): The collection name.

- createCollection (optional): If this parameter has a value of  true  or  yes , then the collection is created if it does not yet exist. Other values will be ignored so the collection must be present for the operation to succeed.

- createCollectionType (optional): If this parameter has a value of  document  or  edge , it will determine the type of collection that is going to be created when the  createCollection  option is set to  true . The default value is  document .

- overwrite (optional): If this parameter has a value of  true  or  yes , then all data in the collection will be removed prior to the import. Note that any existing index definitions will be preseved.

- waitForSync (optional): Wait until documents have been synced to disk before returning.

- onDuplicate (optional): Controls what action is carried out in case of a unique key constraint violation. Possible values are:

- error: this will not import the current document because of the unique key constraint violation. This is the default setting.

- update: this will update an existing document in the database with the data specified in the request. Attributes of the existing document that are not present in the request will be preseved.

- replace: this will replace an existing document in the database with the data specified in the request.

- ignore: this will not update an existing document and simply ignore the error caused by the unique key constraint violation.

Note that update, replace and ignore will only work when the import document in the request contains the _key attribute. update and replace may also fail because of secondary unique key constraint violations.

- complete (optional): If set to  true  or  yes , it will make the whole import fail if any error occurs. Otherwise the import will continue even if some documents cannot be imported.

- details (optional): If set to  true  or  yes , the result will include an attribute  details  with details about documents that could not be imported.

NOTE Swagger examples won't work due to the anchor.

Creates documents in the collection identified by  collection-name . The first line of the request body must contain a JSON-encoded array of attribute names. All following lines in the request body must contain JSON-encoded arrays of attribute values. Each line is interpreted as a separate document, and the values specified will be mapped to the array of attribute names specified in the first header line.

The response is a JSON object with the following attributes:

- created : number of documents imported.

- errors : number of documents that were not imported due to an error.

- empty : number of empty lines found in the input (will only contain a value greater zero for types  documents  or  auto ).

- updated : number of updated/replaced documents (in case  onDuplicate  was set to either  update  or  replace ).

- ignored : number of failed but ignored insert operations (in case  onDuplicate  was set to  ignore ).

- details : if query parameter  details  is set to true, the result will contain a  details  attribute which is an array with more detailed information about which documents could not be inserted.

Note: this API is currently not supported on cluster coordinators.

Return Codes

- 201: is returned if all documents could be imported successfully.

- 400: is returned if  type  contains an invalid value, no  collection  is specified, the documents are incorrectly encoded, or the request is malformed.

- 404: is returned if  collection  or the  _from  or  _to  attributes of an imported edge refer to an unknown collection.

- 409: is returned if the import would trigger a unique key violation and  complete  is set to  true .

- 500: is returned if the server cannot auto-generate a document key (out of keys error) for a document with no user-defined key.

- 501: The server will respond with HTTP 501 if this API is called on a cluster coordinator.

Examples

Importing two documents, with attributes  _key ,  value1  and  value2  each. One line in the import data is empty

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/import?collection=products <<EOF

[ "_key", "value1", "value2" ]

[ "abc", 25, "test" ]

[ "foo", "bar", "baz" ]

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Importing two documents into a new collection

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&createCollection=true <<EOF

[ "value1", "value2" ]

[ 1234, null ]

[ "foo", "bar" ]

[ 534.55, true ]

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Importing into an edge collection, with attributes  _from ,  _to  and  name

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/import?collection=links <<EOF

[ "_from", "_to", "name" ]

[ "products/123", "products/234", "some name" ]

[ "products/332", "products/abc", "other name" ]

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Importing into an edge collection, omitting  _from  or  _to

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/import?collection=links&details=true <<EOF

[ "name" ]

[ "some name" ]

[ "other name" ]

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Violating a unique constraint, but allow partial imports

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&details=true <<EOF

[ "_key", "value1", "value2" ]

[ "abc", 25, "test" ]

[ "abc", "bar", "baz" ]

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Violating a unique constraint, not allowing partial imports

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&complete=true <<EOF

[ "_key", "value1", "value2" ]

[ "abc", 25, "test" ]

[ "abc", "bar", "baz" ]

EOF

HTTP/1.1 409 Conflict

content-type: application/json; charset=utf-8

show response body

Using a non-existing collection

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/import?collection=products <<EOF

[ "_key", "value1", "value2" ]

[ "abc", 25, "test" ]

[ "foo", "bar", "baz" ]

EOF

HTTP/1.1 404 Not Found

content-type: application/json; charset=utf-8

show response body

Using a malformed body

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/import?collection=products <<EOF

{ "_key": "foo", "value1": "bar" }

EOF

HTTP/1.1 400 Bad Request

content-type: application/json; charset=utf-8

show response body

imports documents from JSON

imports documents from JSON

POST /_api/import#json

Post Body documents (required): The body must either be a JSON-encoded array of objects or a string with multiple JSON objects separated by newlines.

Query Parameters

- type (required): Determines how the body of the request will be interpreted.  type  can have the following values:

- documents : when this type is used, each line in the request body is expected to be an individual JSON-encoded document. Multiple JSON objects in the request body need to be separated by newlines.

- list : when this type is used, the request body must contain a single JSON-encoded array of individual objects to import.

- auto : if set, this will automatically determine the body type (either  documents  or  list ).

- collection (required): The collection name.

- createCollection (optional): If this parameter has a value of  true  or  yes , then the collection is created if it does not yet exist. Other values will be ignored so the collection must be present for the operation to succeed.

- createCollectionType (optional): If this parameter has a value of  document  or  edge , it will determine the type of collection that is going to be created when the  createCollection  option is set to  true . The default value is  document .

- fromPrefix (optional): An optional prefix for the values in  _from  attributes. If specified, the value is automatically prepended to each  _from  input value. This allows specifying just the keys for  _from .

- toPrefix (optional): An optional prefix for the values in  _to  attributes. If specified, the value is automatically prepended to each  _to  input value. This allows specifying just the keys for  _to .

- overwrite (optional): If this parameter has a value of  true  or  yes , then all data in the collection will be removed prior to the import. Note that any existing index definitions will be preseved.

- waitForSync (optional): Wait until documents have been synced to disk before returning.

- onDuplicate (optional): Controls what action is carried out in case of a unique key constraint violation. Possible values are:

- error: this will not import the current document because of the unique key constraint violation. This is the default setting.

- update: this will update an existing document in the database with the data specified in the request. Attributes of the existing document that are not present in the request will be preseved.

- replace: this will replace an existing document in the database with the data specified in the request.

- ignore: this will not update an existing document and simply ignore the error caused by a unique key constraint violation.

Note that that update, replace and ignore will only work when the import document in the request contains the _key attribute. update and replace may also fail because of secondary unique key constraint violations.

- complete (optional): If set to  true  or  yes , it will make the whole import fail if any error occurs. Otherwise the import will continue even if some documents cannot be imported.

- details (optional): If set to  true  or  yes , the result will include an attribute  details  with details about documents that could not be imported.

NOTE Swagger examples won't work due to the anchor.

Creates documents in the collection identified by  collection-name . The JSON representations of the documents must be passed as the body of the POST request. The request body can either consist of multiple lines, with each line being a single stand-alone JSON object, or a singe JSON array with sub-objects.

The response is a JSON object with the following attributes:

- created : number of documents imported.

- errors : number of documents that were not imported due to an error.

- empty : number of empty lines found in the input (will only contain a value greater zero for types  documents  or  auto ).

- updated : number of updated/replaced documents (in case  onDuplicate  was set to either  update  or  replace ).

- ignored : number of failed but ignored insert operations (in case  onDuplicate  was set to  ignore ).

- details : if query parameter  details  is set to true, the result will contain a  details  attribute which is an array with more detailed information about which documents could not be inserted.

Note: this API is currently not supported on cluster coordinators.

Return Codes

- 201: is returned if all documents could be imported successfully.

- 400: is returned if  type  contains an invalid value, no  collection  is specified, the documents are incorrectly encoded, or the request is malformed.

- 404: is returned if  collection  or the  _from  or  _to  attributes of an imported edge refer to an unknown collection.

- 409: is returned if the import would trigger a unique key violation and  complete  is set to  true .

- 500: is returned if the server cannot auto-generate a document key (out of keys error) for a document with no user-defined key.

- 501: The server will respond with HTTP 501 if this API is called on a cluster coordinator.

Examples

Importing documents with heterogenous attributes from a JSON array

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=list <<EOF

[

{

"_key" : "abc",

"value1" : 25,

"value2" : "test",

"allowed" : true

},

{

"_key" : "foo",

"name" : "baz"

},

{

"name" : {

"detailed" : "detailed name",

"short" : "short name"

}

}

]

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Importing documents from individual JSON lines

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=documents <<EOF

{ "_key": "abc", "value1": 25, "value2": "test", "allowed": true }

{ "_key": "foo", "name": "baz" }

{ "name": { "detailed": "detailed name", "short": "short name" } }

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Using the auto type detection

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=auto <<EOF

[

{

"_key" : "abc",

"value1" : 25,

"value2" : "test",

"allowed" : true

},

{

"_key" : "foo",

"name" : "baz"

},

{

"name" : {

"detailed" : "detailed name",

"short" : "short name"

}

}

]

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Importing documents into a new collection from a JSON array

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&createCollection=true&type=list <<EOF

[

{

"id" : "12553",

"active" : true

},

{

"id" : "4433",

"active" : false

},

{

"id" : "55932",

"count" : 4334

}

]

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Importing into an edge collection, with attributes  _from ,  _to  and  name

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/import?collection=links&type=documents <<EOF

{ "_from": "products/123", "_to": "products/234" }

{ "_from": "products/332", "_to": "products/abc", "name": "other name" }

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Importing into an edge collection, omitting  _from  or  _to

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/import?collection=links&type=list&details=true <<EOF

[

{

"name" : "some name"

}

]

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Violating a unique constraint, but allow partial imports

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=documents&details=true <<EOF

{ "_key": "abc", "value1": 25, "value2": "test" }

{ "_key": "abc", "value1": "bar", "value2": "baz" }

EOF

HTTP/1.1 201 Created

content-type: application/json; charset=utf-8

show response body

Violating a unique constraint, not allowing partial imports

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=documents&complete=true <<EOF

{ "_key": "abc", "value1": 25, "value2": "test" }

{ "_key": "abc", "value1": "bar", "value2": "baz" }

EOF

HTTP/1.1 409 Conflict

content-type: application/json; charset=utf-8

show response body

Using a non-existing collection

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=documents <<EOF

{ "name": "test" }

EOF

HTTP/1.1 404 Not Found

content-type: application/json; charset=utf-8

show response body

Using a malformed body

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=list <<EOF

{ }

EOF

HTTP/1.1 400 Bad Request

content-type: application/json; charset=utf-8

show response body

Importing Self-Contained JSON Documents

This import method allows uploading self-contained JSON documents. The documents must be uploaded in the body of the HTTP POST request. Each line of the body will be interpreted as one stand-alone document. Empty lines in the body are allowed but will be skipped. Using this format, the documents are imported line-wise.

Example input data: { "_key": "key1", ... } { "_key": "key2", ... } ...

To use this method, the type query parameter should be set to documents.

It is also possible to upload self-contained JSON documents that are embedded into a JSON array. Each element from the array will be treated as a document and be imported.

Example input data for this case:

[

{ "_key": "key1", ... },

{ "_key": "key2", ... },

...

]

This format does not require each document to be on a separate line, and any whitespace in the JSON data is allowed. It can be used to import a JSON-formatted result array (e.g. from arangosh) back into ArangoDB. Using this format requires ArangoDB to parse the complete array and keep it in memory for the duration of the import. This might be more resource-intensive than the line-wise processing.

To use this method, the type query parameter should be set to array.

Setting the type query parameter to auto will make the server auto-detect whether the data are line-wise JSON documents (type = documents) or a JSON array (type = array).

Examples

curl --data-binary @- -X POST --dump - "http://localhost:8529/_api/import?type=documents&collection=test&createCollection=true"

{ "name" : "test", "gender" : "male", "age" : 39 }

{ "type" : "bird", "name" : "robin" }

HTTP/1.1 201 Created

Server: ArangoDB

Connection: Keep-Alive

Content-type: application/json; charset=utf-8

{"error":false,"created":2,"empty":0,"errors":0}

The server will respond with an HTTP 201 if everything went well. The number of documents imported will be returned in the created attribute of the response. If any documents were skipped or incorrectly formatted, this will be returned in the errors attribute. There will also be an attribute empty in the response, which will contain a value of 0.

If the details parameter was set to true in the request, the response will also contain an attribute details which is an array of details about errors that occurred on the server side during the import. This array might be empty if no errors occurred.

Importing Headers and Values

When using this type of import, the attribute names of the documents to be imported are specified separate from the actual document value data. The first line of the HTTP POST request body must be a JSON array containing the attribute names for the documents that follow. The following lines are interpreted as the document data. Each document must be a JSON array of values. No attribute names are needed or allowed in this data section.

Examples

curl --data-binary @- -X POST --dump - "http://localhost:8529/_api/import?collection=test&createCollection=true"

[ "firstName", "lastName", "age", "gender" ]

[ "Joe", "Public", 42, "male" ]

[ "Jane", "Doe", 31, "female" ]

HTTP/1.1 201 Created

Server: ArangoDB

Connection: Keep-Alive

Content-type: application/json; charset=utf-8

{"error":false,"created":2,"empty":0,"errors":0}

The server will again respond with an HTTP 201 if everything went well. The number of documents imported will be returned in the created attribute of the response. If any documents were skipped or incorrectly formatted, this will be returned in the errors attribute. The number of empty lines in the input file will be returned in the empty attribute.

If the details parameter was set to true in the request, the response will also contain an attribute details which is an array of details about errors that occurred on the server side during the import. This array might be empty if no errors occurred.

Importing into Edge Collections

Please note that when importing documents into an edge collection, it is mandatory that all imported documents contain the _from and _to attributes, and that these contain references to existing collections.

Please also note that it is not possible to create a new edge collection on the fly using the createCollection parameter.

HTTP Interface for Batch Requests

Clients normally send individual operations to ArangoDB in individual HTTP requests. This is straightforward and simple, but has the disadvantage that the network overhead can be significant if many small requests are issued in a row.

To mitigate this problem, ArangoDB offers a batch request API that clients can use to send multiple operations in one batch to ArangoDB. This method is especially useful when the client has to send many HTTP requests with a small body/payload and the individual request results do not depend on each other.

Clients can use ArangoDB's batch API by issuing a multipart HTTP POST request to the URL /_api/batch handler. The handler will accept the request if the Content-type is multipart/form-data and a boundary string is specified. ArangoDB will then decompose the batch request into its individual parts using this boundary. This also means that the boundary string itself must not be contained in any of the parts. When ArangoDB has split the multipart request into its individual parts, it will process all parts sequentially as if it were a standalone request. When all parts are processed, ArangoDB will generate a multipart HTTP response that contains one part for each part operation result. For example, if you send a multipart request with 5 parts, ArangoDB will send back a multipart response with 5 parts as well.

The server expects each part message to start with exactly the following "header":

Content-type: application/x-arango-batchpart

You can optionally specify a Content-Id "header" to uniquely identify each part message. The server will return the Content-Id in its response if it is specified. Otherwise, the server will not send a Content-Id "header" back. The server will not validate the uniqueness of the Content-Id. After the mandatory Content-type and the optional Content-Id header, two Windows line breaks (i.e. \r\n\r\n) must follow. Any deviation of this structure might lead to the part being rejected or incorrectly interpreted. The part request payload, formatted as a regular HTTP request, must follow the two Windows line breaks literal directly.

Note that the literal Content-type: application/x-arango-batchpart technically is the header of the MIME part, and the HTTP request (including its headers) is the body part of the MIME part.

An actual part request should start with the HTTP method, the called URL, and the HTTP protocol version as usual, followed by arbitrary HTTP headers. Its body should follow after the usual \r\n\r\n literal. Part requests are therefore regular HTTP requests, only embedded inside a multipart message.

The following example will send a batch with 3 individual document creation operations. The boundary used in this example is XXXsubpartXXX.

Examples

> curl -X POST --data-binary @- --header "Content-type: multipart/form-data; boundary=XXXsubpartXXX" http://localhost:8529/_api/batch

--XXXsubpartXXX

Content-type: application/x-arango-batchpart

Content-Id: 1

POST /_api/document?collection=xyz&createCollection=true HTTP/1.1

{"a":1,"b":2,"c":3}

--XXXsubpartXXX

Content-type: application/x-arango-batchpart

Content-Id: 2

POST /_api/document?collection=xyz HTTP/1.1

{"a":1,"b":2,"c":3,"d":4}

--XXXsubpartXXX

Content-type: application/x-arango-batchpart

Content-Id: 3

POST /_api/document?collection=xyz HTTP/1.1

{"a":1,"b":2,"c":3,"d":4,"e":5}

--XXXsubpartXXX--

The server will then respond with one multipart message, containing the overall status and the individual results for the part operations. The overall status should be 200 except there was an error while inspecting and processing the multipart message. The overall status therefore does not indicate the success of each part operation, but only indicates whether the multipart message could be handled successfully.

Each part operation will return its own status value. As the part operation results are regular HTTP responses (just included in one multipart response), the part operation status is returned as a HTTP status code. The status codes of the part operations are exactly the same as if you called the individual operations standalone. Each part operation might also return arbitrary HTTP headers and a body/payload:

Examples

HTTP/1.1 200 OK

Connection: Keep-Alive

Content-type: multipart/form-data; boundary=XXXsubpartXXX

Content-length: 1055

--XXXsubpartXXX

Content-type: application/x-arango-batchpart

Content-Id: 1

HTTP/1.1 202 Accepted

Content-type: application/json; charset=utf-8

Etag: "9514299"

Content-length: 53

{"error":false,"_id":"xyz/9514299","_key":"9514299","_rev":"9514299"}

--XXXsubpartXXX

Content-type: application/x-arango-batchpart

Content-Id: 2

HTTP/1.1 202 Accepted

Content-type: application/json; charset=utf-8

Etag: "9579835"

Content-length: 53

{"error":false,"_id":"xyz/9579835","_key":"9579835","_rev":"9579835"}

--XXXsubpartXXX

Content-type: application/x-arango-batchpart

Content-Id: 3

HTTP/1.1 202 Accepted

Content-type: application/json; charset=utf-8

Etag: "9645371"

Content-length: 53

{"error":false,"_id":"xyz/9645371","_key":"9645371","_rev":"9645371"}

--XXXsubpartXXX--

In the above example, the server returned an overall status code of 200, and each part response contains its own status value (202 in the example):

When constructing the multipart HTTP response, the server will use the same boundary that the client supplied. If any of the part responses has a status code of 400 or greater, the server will also return an HTTP header x-arango-errors containing the overall number of part requests that produced errors:

Examples

> curl -X POST --data-binary @- --header "Content-type: multipart/form-data; boundary=XXXsubpartXXX" http://localhost:8529/_api/batch

--XXXsubpartXXX

Content-type: application/x-arango-batchpart

POST /_api/document?collection=nonexisting

{"a":1,"b":2,"c":3}

--XXXsubpartXXX

Content-type: application/x-arango-batchpart

POST /_api/document?collection=xyz

{"a":1,"b":2,"c":3,"d":4}

--XXXsubpartXXX--

In this example, the overall response code is 200, but as some of the part request failed (with status code 404), the x-arango-errors header of the overall response is 1:

Examples

HTTP/1.1 200 OK

x-arango-errors: 1

Content-type: multipart/form-data; boundary=XXXsubpartXXX

Content-length: 711

--XXXsubpartXXX

Content-type: application/x-arango-batchpart

HTTP/1.1 404 Not Found

Content-type: application/json; charset=utf-8

Content-length: 111

{"error":true,"code":404,"errorNum":1203,"errorMessage":"collection \/_api\/collection\/nonexisting not found"}

--XXXsubpartXXX

Content-type: application/x-arango-batchpart

HTTP/1.1 202 Accepted

Content-type: application/json; charset=utf-8

Etag: "9841979"

Content-length: 53

{"error":false,"_id":"xyz/9841979","_key":"9841979","_rev":"9841979"}

--XXXsubpartXXX--

Please note that the database used for all part operations of a batch request is determined by scanning the original URL (the URL that contains /_api/batch). It is not possible to override the database name in part operations of a batch. When doing so, any other database name used in a batch part will be ignored.

executes a batch request

executes a batch request

POST /_api/batch

Post Body body (required): The multipart batch request, consisting of the envelope and the individual batch parts.

Executes a batch request. A batch request can contain any number of other requests that can be sent to ArangoDB in isolation. The benefit of using batch requests is that batching requests requires less client/server roundtrips than when sending isolated requests.

All parts of a batch request are executed serially on the server. The server will return the results of all parts in a single response when all parts are finished.

Technically, a batch request is a multipart HTTP request, with content-type  multipart/form-data . A batch request consists of an envelope and the individual batch part actions. Batch part actions are "regular" HTTP requests, including full header and an optional body. Multiple batch parts are separated by a boundary identifier. The boundary identifier is declared in the batch envelope. The MIME content-type for each individual batch part must be  application/x-arango-batchpart .

Please note that when constructing the individual batch parts, you must use CRLF (   ) as the line terminator as in regular HTTP messages.

The response sent by the server will be an  HTTP 200  response, with an optional error summary header  x-arango-errors . This header contains the number of batch part operations that failed with an HTTP error code of at least 400. This header is only present in the response if the number of errors is greater than zero.

The response sent by the server is a multipart response, too. It contains the individual HTTP responses for all batch parts, including the full HTTP result header (with status code and other potential headers) and an optional result body. The individual batch parts in the result are seperated using the same boundary value as specified in the request.

The order of batch parts in the response will be the same as in the original client request. Client can additionally use the  Content-Id  MIME header in a batch part to define an individual id for each batch part. The server will return this id is the batch part responses, too.

Return Codes

- 200: is returned if the batch was received successfully. HTTP 200 is returned even if one or multiple batch part actions failed.

- 400: is returned if the batch envelope is malformed or incorrectly formatted. This code will also be returned if the content-type of the overall batch request or the individual MIME parts is not as expected.

- 405: is returned when an invalid HTTP method is used.

Examples

Sending a batch request with five batch parts:

- GET /_api/version

- DELETE /_api/collection/products

- POST /_api/collection/products

- GET /_api/collection/products/figures

- DELETE /_api/collection/products

The boundary ( SomeBoundaryValue ) is passed to the server in the HTTP  Content-Type  HTTP header. Please note the reply is not displayed all accurate.

shell> curl -X POST --header 'Content-Type: multipart/form-data; boundary=SomeBoundaryValue' --data-binary @- --dump - http://localhost:8529/_api/batch <<EOF

--SomeBoundaryValue

Content-Type: application/x-arango-batchpart

Content-Id: myId1

GET /_api/version HTTP/1.1

--SomeBoundaryValue

Content-Type: application/x-arango-batchpart

Content-Id: myId2

DELETE /_api/collection/products HTTP/1.1

--SomeBoundaryValue

Content-Type: application/x-arango-batchpart

Content-Id: someId

POST /_api/collection/products HTTP/1.1

{ "name": "products" }

--SomeBoundaryValue

Content-Type: application/x-arango-batchpart

Content-Id: nextId

GET /_api/collection/products/figures HTTP/1.1

--SomeBoundaryValue

Content-Type: application/x-arango-batchpart

Content-Id: otherId

DELETE /_api/collection/products HTTP/1.1

--SomeBoundaryValue--

EOF

HTTP/1.1 200 OK

content-type: multipart/form-data; boundary=SomeBoundaryValue

x-arango-errors: 1

"--SomeBoundaryValue

Content-Type: application/x-arango-batchpart

Content-Id: myId1

HTTP/1.1 200 OK

Content-Type: application/json; charset=utf-8

Content-Length: 37

{\"server\":\"arango\",\"version\":\"2.8.9\"}

--SomeBoundaryValue

Content-Type: application/x-arango-batchpart

Content-Id: myId2

HTTP/1.1 404 Not Found

Content-Type: application/json; charset=utf-8

Content-Length: 88

{\"error\":true,\"code\":404,\"errorNum\":1203,\"errorMessage\":\"unknown collection 'products'\"}

--SomeBoundaryValue

Content-Type: application/x-arango-batchpart

Content-Id: someId

HTTP/1.1 200 OK

Location: /_db/_system/_api/collection/products

Content-Type: application/json; charset=utf-8

Content-Length: 137

{\"id\":\"643363048\",\"name\":\"products\",\"waitForSync\":false,\"isVolatile\":false,\"isSystem\":false,\"status\":3,\"type\":2,\"error\":false,\"code\":200}

--SomeBoundaryValue

Content-Type: application/x-arango-batchpart

Content-Id: nextId

HTTP/1.1 200 OK

Location: /_db/_system/_api/collection/products/figures

Content-Type: application/json; charset=utf-8

Content-Length: 765

{\"id\":\"643363048\",\"name\":\"products\",\"isSystem\":false,\"doCompact\":true,\"isVolatile\":false,\"journalSize\":1048576,\"keyOptions\":{\"type\":\"traditional\",\"allowUserKeys\":true},\"waitForSync\":false,\"indexBuckets\":8,\"count\":0,\"figures\":{\"alive\":{\"count\":0,\"size\":0},\"dead\":{\"count\":0,\"size\":0,\"deletion\":0},\"datafiles\":{\"count\":0,\"fileSize\":0},\"journals\":{\"count\":0,\"fileSize\":0},\"compactors\":{\"count\":0,\"fileSize\":0},\"shapefiles\":{\"count\":0,\"fileSize\":0},\"shapes\":{\"count\":0,\"size\":0},\"attributes\":{\"count\":0,\"size\":0},\"indexes\":{\"count\":1,\"size\":16064},\"lastTick\":\"0\",\"uncollectedLogfileEntries\":0,\"documentReferences\":0,\"waitingFor\":\"-\",\"compactionStatus\":{\"message\":\"compaction not yet started\",\"time\":\"2016-05-22T19:05:44Z\"}},\"status\":3,\"type\":2,\"error\":false,\"code\":200}

--SomeBoundaryValue

Content-Type: application/x-arango-batchpart

Content-Id: otherId

HTTP/1.1 200 OK

Content-Type: application/json; charset=utf-8

Content-Length: 43

{\"id\":\"643363048\",\"error\":false,\"code\":200}

--SomeBoundaryValue--"

Sending a batch request, setting the boundary implicitly (the server will in this case try to find the boundary at the beginning of the request body).

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/batch <<EOF

--SomeBoundaryValue

Content-Type: application/x-arango-batchpart

DELETE /_api/collection/notexisting1 HTTP/1.1

--SomeBoundaryValue

Content-Type: application/x-arango-batchpart

DELETE /_api/collection/notexisting2 HTTP/1.1

--SomeBoundaryValue--

EOF

HTTP/1.1 200 OK

x-arango-errors: 2

"--SomeBoundaryValue

Content-Type: application/x-arango-batchpart

HTTP/1.1 404 Not Found

Content-Type: application/json; charset=utf-8

Content-Length: 92

{\"error\":true,\"code\":404,\"errorNum\":1203,\"errorMessage\":\"unknown collection 'notexisting1'\"}

--SomeBoundaryValue

Content-Type: application/x-arango-batchpart

HTTP/1.1 404 Not Found

Content-Type: application/json; charset=utf-8

Content-Length: 92

{\"error\":true,\"code\":404,\"errorNum\":1203,\"errorMessage\":\"unknown collection 'notexisting2'\"}

--SomeBoundaryValue--"

Deletes an existing dump batch

handle a dump batch command

DELETE /_api/replication/batch/{id}

Note: These calls are uninteresting to users.

Path Parameters

- id (required): The id of the batch.

Deletes the existing dump batch, allowing compaction and cleanup to resume.

Note: on a coordinator, this request must have the query parameter DBserver which must be an ID of a DBserver. The very same request is forwarded synchronously to that DBserver. It is an error if this attribute is not bound in the coordinator case.

Return Codes

- 204: is returned if the batch was deleted successfully.

- 400: is returned if the batch was not found.

- 405: is returned when an invalid HTTP method is used.

Create new dump batch

handle a dump batch command

POST /_api/replication/batch

Note: These calls are uninteresting to users.

Body Parameters

- ttl: the time-to-live for the new batch (in seconds) A JSON object with the batch configuration.

A JSON object with the batch configuration.

Creates a new dump batch and returns the batch's id.

The response is a JSON object with the following attributes:

- id: the id of the batch

Note: on a coordinator, this request must have the query parameter DBserver which must be an ID of a DBserver. The very same request is forwarded synchronously to that DBserver. It is an error if this attribute is not bound in the coordinator case.

Return Codes

- 204: is returned if the batch was created successfully.

- 400: is returned if the ttl value is invalid or if DBserver attribute is not specified or illegal on a coordinator.

- 405: is returned when an invalid HTTP method is used.

Prolong existing dump batch

handle a dump batch command

PUT /_api/replication/batch/{id}

Note: These calls are uninteresting to users.

Body Parameters

- ttl: the time-to-live for the new batch (in seconds)

Path Parameters

- id (required): The id of the batch.

Extends the ttl of an existing dump batch, using the batch's id and the provided ttl value.

If the batch's ttl can be extended successfully, the response is empty.

Note: on a coordinator, this request must have the query parameter DBserver which must be an ID of a DBserver. The very same request is forwarded synchronously to that DBserver. It is an error if this attribute is not bound in the coordinator case.

Return Codes

- 204: is returned if the batch's ttl was extended successfully.

- 400: is returned if the ttl value is invalid or the batch was not found.

- 405: is returned when an invalid HTTP method is used.

Arangoimp

This manual describes the ArangoDB importer arangoimp, which can be used for bulk imports.

The most convenient method to import a lot of data into ArangoDB is to use the arangoimp command-line tool. It allows you to import data records from a file into an existing database collection.

It is possible to import document keys with the documents using the _key attribute. When importing into an edge collection, it is mandatory that all imported documents have the _from and _to attributes, and that they contain valid references.

Let's assume for the following examples you want to import user data into an existing collection named "users" on the server.

Importing Data into an ArangoDB Database

Importing JSON-encoded Data

Let's further assume the import at hand is encoded in JSON. We'll be using these example user records to import:

{ "name" : { "first" : "John", "last" : "Connor" }, "active" : true, "age" : 25, "likes" : [ "swimming"] }

{ "name" : { "first" : "Jim", "last" : "O'Brady" }, "age" : 19, "likes" : [ "hiking", "singing" ] }

{ "name" : { "first" : "Lisa", "last" : "Jones" }, "dob" : "1981-04-09", "likes" : [ "running" ] }

To import these records, all you need to do is to put them into a file (with one line for each record to import) and run the following command:

> arangoimp --file "data.json" --type json --collection "users"

This will transfer the data to the server, import the records, and print a status summary. To show the intermediate progress during the import process, the option --progress can be added. This option will show the percentage of the input file that has been sent to the server. This will only be useful for big import files.

> arangoimp --file "data.json" --type json --collection users --progress true

It is also possible to use the output of another command as an input for arangoimp. For example, the following shell command can be used to pipe data from the  cat  process to arangoimp:

> cat data.json | arangoimp --file - --type json --collection users

Note that you have to use  --file -  if you want to use another command as input for arangoimp. No progress can be reported for such imports as the size of the input will be unknown to arangoimp.

By default, the endpoint tcp://127.0.0.1:8529 will be used. If you want to specify a different endpoint, you can use the --server.endpoint option. You probably want to specify a database user and password as well. You can do so by using the options --server.username and --server.password. If you do not specify a password, you will be prompted for one.

> arangoimp --server.endpoint tcp://127.0.0.1:8529 --server.username root --file "data.json" --type json --collection "users"

Note that the collection (users in this case) must already exist or the import will fail. If you want to create a new collection with the import data, you need to specify the --create-collection option. Note that by default it will create a document collection and no ede collection.

> arangoimp --file "data.json" --type json --collection "users" --create-collection true

To create an edge collection instead, use the --create-collection-type option and set it to edge:

> arangoimp --file "data.json" --collection "myedges" --create-collection true --create-collection-type edge

When importing data into an existing collection it is often convenient to first remove all data from the collection and then start the import. This can be achieved by passing the --overwrite parameter to arangoimp. If it is set to true, any existing data in the collection will be removed prior to the import. Note that any existing index definitions for the collection will be preserved even if --overwrite is set to true.

> arangoimp --file "data.json" --type json --collection "users" --overwrite true

As the import file already contains the data in JSON format, attribute names and data types are fully preserved. As can be seen in the example data, there is no need for all data records to have the same attribute names or types. Records can be inhomogeneous.

Please note that by default, arangoimp will import data into the specified collection in the default database (_system). To specify a different database, use the --server.database option when invoking arangoimp.

JSON input file formats

Note: arangoimp supports two formats when importing JSON data from a file. The first format (also used above) requires the input file to contain one complete JSON document in each line, e.g.

{ "_key": "one", "value": 1 }

{ "_key": "two", "value": 2 }

{ "_key": "foo", "value": "bar" }

...

The above format can be imported sequentially by arangoimp. It will read data from the input file in chunks and send it in batches to the server. Each batch will be about as big as specified in the command-line parameter --batch-size.

An alternative is to put one big JSON document into the input file like this:

[

{ "_key": "one", "value": 1 },

{ "_key": "two", "value": 2 },

{ "_key": "foo", "value": "bar" },

...

]

This format allows line breaks within the input file as required. The downside is that the whole input file will need to be read by arangoimp before it can send the first batch. This might be a problem if the input file is big. By default, arangoimp will allow importing such files up to a size of about 16 MB.

If you want to allow your arangoimp instance to use more memory, you may want to increase the maximum file size by specifying the command-line option --batch-size. For example, to set the batch size to 32 MB, use the following command:

> arangoimp --file "data.json" --type json --collection "users" --batch-size 33554432

Please also note that you may need to increase the value of --batch-size if a single document inside the input file is bigger than the value of --batch-size.

Importing CSV Data

arangoimp also offers the possibility to import data from CSV files. This comes handy when the data at hand is in CSV format already and you don't want to spend time converting them to JSON for the import.

To import data from a CSV file, make sure your file contains the attribute names in the first row. All the following lines in the file will be interpreted as data records and will be imported.

The CSV import requires the data to have a homogeneous structure. All records must have exactly the same amount of columns as there are headers.

The cell values can have different data types though. If a cell does not have any value, it can be left empty in the file. These values will not be imported so the attributes will not "be there" in document created. Values enclosed in quotes will be imported as strings, so to import numeric values, boolean values or the null value, don't enclose the value in quotes in your file.

We'll be using the following import for the CSV import:

"first","last","age","active","dob"

"John","Connor",25,true,

"Jim","O'Brady",19,,

"Lisa","Jones",,,"1981-04-09"

Hans,dos Santos,0123,,

Wayne,Brewer,,false,

The command line to execute the import is:

> arangoimp --file "data.csv" --type csv --collection "users"

The above data will be imported into 5 documents which will look as follows:

{ "first" : "John", "last" : "Connor", "active" : true, "age" : 25 }

{ "first" : "Jim", "last" : "O'Brady", "age" : 19 }

{ "first" : "Lisa", "last" : "Jones", "dob" : "1981-04-09" }

{ "first" : "Hans", "last" : "dos Santos", "age" : 123 }

{ "first" : "Wayne", "last" : "Brewer", "active" : false }

As can be seen, values left completely empty in the input file will be treated as absent. Numeric values not enclosed in quotes will be treated as numbers. Note that leading zeros in numeric values will be removed. To import numbers with leading zeros, please use strings. The literals true and false will be treated as booleans if they are not enclosed in quotes. Other values not enclosed in quotes will be treated as strings. Any values enclosed in quotes will be treated as strings, too.

String values containing the quote character or the separator must be enclosed with quote characters. Within a string, the quote character itself must be escaped with another quote character (or with a backslash if the --backslash-escape option is used).

Note that the quote and separator characters can be adjusted via the --quote and --separator arguments when invoking arangoimp. The quote character defaults to the double quote ("). To use a literal quote in a string, you can use two quote characters. To use backslash for escaping quote characters, please set the option --backslash-escape to true.

The importer supports Windows (CRLF) and Unix (LF) line breaks. Line breaks might also occur inside values that are enclosed with the quote character.

Here's an example for using literal quotes and newlines inside values:

"name","password"

"Foo","r4ndom""123!"

"Bar","wow!

this is a

multine password!"

"Bartholomew ""Bart"" Simpson","Milhouse"

Extra whitespace at the end of each line will be ignored. Whitespace at the start of lines or between field values will not be ignored, so please make sure that there is no extra whitespace in front of values or between them.

Importing TSV Data

You may also import tab-separated values (TSV) from a file. This format is very simple: every line in the file represents a data record. There is no quoting or escaping. That also means that the separator character (which defaults to the tabstop symbol) must not be used anywhere in the actual data.

As with CSV, the first line in the TSV file must contain the attribute names, and all lines must have an identical number of values.

If a different separator character or string should be used, it can be specified with the --separator argument.

An example command line to execute the TSV import is:

> arangoimp --file "data.tsv" --type tsv --collection "users"

Importing into an Edge Collection

arangoimp can also be used to import data into an existing edge collection. The import data must, for each edge to import, contain at least the _from and _to attributes. These indicate which other two documents the edge should connect. It is necessary that these attributes are set for all records, and point to valid document ids in existing collections.

Examples

{ "_from" : "users/1234", "_to" : "users/4321", "desc" : "1234 is connected to 4321" }

Note: The edge collection must already exist when the import is started. Using the --create-collection flag will not work because arangoimp will always try to create a regular document collection if the target collection does not exist.

Updating existing documents

By default, arangoimp will try to insert all documents from the import file into the specified collection. In case the import file contains documents that are already present in the target collection (matching is done via the _key attributes), then a default arangoimp run will not import these documents and complain about unique key constraint violations.

However, arangoimp can be used to update or replace existing documents in case they already exist in the target collection. It provides the command-line option --on-duplicate to control the behavior in case a document is already present in the database.

The default value of --on-duplicate is error. This means that when the import file contains a document that is present in the target collection already, then trying to re-insert a document with the same _key value is considered an error, and the document in the database will not be modified.

Other possible values for --on-duplicate are:

- update: each document present in the import file that is also present in the target collection already will be updated by arangoimp. update will perform a partial update of the existing document, modifying only the attributes that are present in the import file and leaving all other attributes untouched.

The values of system attributes _id, _key, _rev, _from and _to cannot be updated or replaced in existing documents.

- replace: each document present in the import file that is also present in the target collection already will be replace by arangoimp. replace will replace the existing document entirely, resulting in a document with only the attributes specified in the import file.

The values of system attributes _id, _key, _rev, _from and _to cannot be updated or replaced in existing documents.

- ignore: each document present in the import file that is also present in the target collection already will be ignored and not modified in the target collection.

When --on-duplicate is set to either update or replace, arangoimp will return the number of documents updated/replaced in the updated return value. When set to another value, the value of updated will always be zero. When --on-duplicate is set to ignore, arangoimp will return the number of ignored documents in the ignored return value. When set to another value, ignored will always be zero.

It is possible to perform a combination of inserts and updates/replaces with a single arangoimp run. When --on-duplicate is set to update or replace, all documents present in the import file will be inserted into the target collection provided they are valid and do not already exist with the specified _key. Documents that are already present in the target collection (identified by _key attribute) will instead be updated/replaced.

Arangoimp result output

An arangoimp import run will print out the final results on the command line. It will show the

- number of documents created (created)

- number of documents updated/replaced (updated/replaced, only non-zero if --on-duplicate was set to update or replace, see below)

- number of warnings or errors that occurred on the server side (warnings/errors)

- number of ignored documents (only non-zero if --on-duplicate was set to ignore).

Example

created:          2

warnings/errors:  0

updated/replaced: 0

ignored:          0

For CSV and TSV imports, the total number of input file lines read will also be printed (lines read).

arangoimp will also print out details about warnings and errors that happened on the server-side (if any).

Attribute Naming and Special Attributes

Attributes whose names start with an underscore are treated in a special way by ArangoDB:

- the optional _key attribute contains the document's key. If specified, the value must be formally valid (e.g. must be a string and conform to the naming conventions). Additionally, the key value must be unique within the collection the import is run for.

- _from: when importing into an edge collection, this attribute contains the id of one of the documents connected by the edge. The value of _from must be a syntactically valid document id and the referred collection must exist.

- _to: when importing into an edge collection, this attribute contains the id of the other document connected by the edge. The value of _to must be a syntactically valid document id and the referred collection must exist.

- _rev: this attribute contains the revision number of a document. However, the revision numbers are managed by ArangoDB and cannot be specified on import. Thus any value in this attribute is ignored on import.

If you import values into _key, you should make sure they are valid and unique.

When importing data into an edge collection, you should make sure that all import documents can _from and _to and that their values point to existing documents.

To avoid specifying complete document ids (consisting of collection names and document keys) for _from and _to values, there are the options --from-collection-prefix and --to-collection-prefix. If specified, these values will be automatically prepended to each value in _from (or _to resp.). This allows specifying only document keys inside _from and/or _to.

Example

> arangoimp --from-collection-prefix users --to-collection-prefix products ...

Importing the following document will then create an edge between users/1234 and products/4321:

{ "_from" : "1234", "_to" : "4321", "desc" : "users/1234 is connected to products/4321" }

Dumping Data from an ArangoDB database

To dump data from an ArangoDB server instance, you will need to invoke arangodump. Dumps can be re-imported with arangorestore. arangodump can be invoked by executing the following command:

unix> arangodump --output-directory "dump"

This will connect to an ArangoDB server and dump all non-system collections from the default database (_system) into an output directory named dump. Invoking arangodump will fail if the output directory already exists. This is an intentional security measure to prevent you from accidentally overwriting already dumped data. If you are positive that you want to overwrite data in the output directory, you can use the parameter --overwrite true to confirm this:

unix> arangodump --output-directory "dump" --overwrite true

arangodump will by default connect to the _system database using the default endpoint. If you want to connect to a different database or a different endpoint, or use authentication, you can use the following command-line options:

- --server.database : name of the database to connect to

- --server.endpoint : endpoint to connect to

- --server.username : username

- --server.password : password to use (omit this and you'll be prompted for the password)

- --server.disable-authentication : whether or not to use authentication

Here's an example of dumping data from a non-standard endpoint, using a dedicated database name:

unix> arangodump --server.endpoint tcp://192.168.173.13:8531 --server.username backup --server.database mydb --output-directory "dump"

When finished, arangodump will print out a summary line with some aggregate statistics about what it did, e.g.:

Processed 43 collection(s), wrote 408173500 byte(s) into datafiles, sent 88 batch(es)

By default, arangodump will dump both structural information and documents from all non-system collections. To adjust this, there are the following command-line arguments:

- --dump-data : set to true to include documents in the dump. Set to false to exclude documents. The default value is true.

- --include-system-collections : whether or not to include system collections in the dump. The default value is false.

For example, to only dump structural information of all collections (including system collections), use:

unix> arangodump --dump-data false --include-system-collections true --output-directory "dump"

To restrict the dump to just specific collections, there is is the --collection option. It can be specified multiple times if required:

unix> arangodump --collection myusers --collection myvalues --output-directory "dump"

Structural information for a collection will be saved in files with name pattern .structure.json. Each structure file will contains a JSON object with these attributes:

- parameters: contains the collection properties

- indexes: contains the collection indexes

Document data for a collection will be saved in files with name pattern .data.json. Each line in a data file is a document insertion/update or deletion marker, alongside with some meta data.

Starting with Version 2.1 of ArangoDB, the arangodump tool also supports sharding. Simply point it to one of the coordinators and it will behave exactly as described above, working on sharded collections in the cluster.

However, as opposed to the single instance situation, this operation does not guarantee to dump a consistent snapshot if write operations happen during the dump operation. It is therefore recommended not to perform any data-modifcation operations on the cluster whilst arangodump is running.

As above, the output will be one structure description file and one data file per sharded collection. Note that the data in the data file is sorted first by shards and within each shard by ascending timestamp. The structural information of the collection contains the number of shards and the shard keys.

Arangorestore

To reload data from a dump previously created with arangodump, ArangoDB provides the arangorestore tool.

Reloading Data into an ArangoDB database

Invoking arangorestore

arangorestore can be invoked from the command-line as follows:

unix> arangorestore --input-directory "dump"

This will connect to an ArangoDB server and reload structural information and documents found in the input directory dump. Please note that the input directory must have been created by running arangodump before.

arangorestore will by default connect to the _system database using the default endpoint. If you want to connect to a different database or a different endpoint, or use authentication, you can use the following command-line options:

- --server.database : name of the database to connect to

- --server.endpoint : endpoint to connect to

- --server.username : username

- --server.password : password to use (omit this and you'll be prompted for the password)

- --server.disable-authentication : whether or not to use authentication

Since version 2.6 arangorestore provides the option --create-database. Setting this option to true will create the target database if it does not exist. When creating the target database, the username and passwords passed to arangorestore (in options --server.username and --server.password) will be used to create an initial user for the new database.

Here's an example of reloading data to a non-standard endpoint, using a dedicated database name:

unix> arangorestore --server.endpoint tcp://192.168.173.13:8531 --server.username backup --server.database mydb --input-directory "dump"

To create the target database whe restoring, use a command like this:

unix> arangorestore --server.username backup --server.database newdb --create-database true --input-directory "dump"

arangorestore will print out its progress while running, and will end with a line showing some aggregate statistics:

Processed 2 collection(s), read 2256 byte(s) from datafiles, sent 2 batch(es)

By default, arangorestore will re-create all non-system collections found in the input directory and load data into them. If the target database already contains collections which are also present in the input directory, the existing collections in the database will be dropped and re-created with the data found in the input directory.

The following parameters are available to adjust this behavior:

- --create-collection : set to true to create collections in the target database. If the target database already contains a collection with the same name, it will be dropped first and then re-created with the properties found in the input directory. Set to false to keep existing collections in the target database. If set to false and arangorestore encounters a collection that is present in both the target database and the input directory, it will abort. The default value is true.

- --import-data : set to true to load document data into the collections in the target database. Set to false to not load any document data. The default value is true.

- --include-system-collections : whether or not to include system collections when re-creating collections or reloading data. The default value is false.

For example, to (re-)create all non-system collections and load document data into them, use:

unix> arangorestore --create-collection true --import-data true --input-directory "dump"

This will drop potentially existing collections in the target database that are also present in the input directory.

To include system collections too, use --include-system-collections true:

unix> arangorestore --create-collection true --import-data true --include-system-collections true --input-directory "dump"

To (re-)create all non-system collections without loading document data, use:

unix> arangorestore --create-collection true --import-data false --input-directory "dump"

This will also drop existing collections in the target database that are also present in the input directory.

To just load document data into all non-system collections, use:

unix> arangorestore --create-collection false --import-data true --input-directory "dump"

To restrict reloading to just specific collections, there is is the --collection option. It can be specified multiple times if required:

unix> arangorestore --collection myusers --collection myvalues --input-directory "dump"

Collections will be processed by in alphabetical order by arangorestore, with all document collections being processed before all edge collections. This is to ensure that reloading data into edge collections will have the document collections linked in edges (_from and _to attributes) loaded.

Restoring Revision Ids and Collection Ids

arangorestore will reload document and edges data with the exact same _key, _from and _to values found in the input directory. However, when loading document data, it will assign its own values for the _rev attribute of the reloaded documents. Though this difference is intentional (normally, every server should create its own _rev values) there might be situations when it is required to re-use the exact same _rev values for the reloaded data. This can be achieved by setting the --recycle-ids parameter to true:

unix> arangorestore --collection myusers --collection myvalues --recycle-ids true --input-directory "dump"

Note that setting --recycle-ids to true will also cause collections to be (re-)created in the target database with the exact same collection id as in the input directory. Any potentially existing collection in the target database with the same collection id will then be dropped.

Setting --recycle-ids to false or omitting it will only use the collection name from the input directory and allow the target database to create the collection with a different id (though with the same name) than in the input directory.

Reloading Data into a different Collection

With some creativity you can use arangodump and arangorestore to transfer data from one collection into another (either on the same server or not). For example, to copy data from a collection myvalues in database mydb into a collection mycopyvalues in database mycopy, you can start with the following command:

unix> arangodump --collection myvalues --server.database mydb --output-directory "dump"

This will create two files, myvalues.structure.json and myvalues.data.json, in the output directory. To load data from the datafile into an existing collection mycopyvalues in database mycopy, rename the files to mycopyvalues.structure.json and mycopyvalues.data.json. After that, run the following command:

unix> arangorestore --collection mycopyvalues --server.database mycopy --input-directory "dump"

Using arangorestore with sharding

As of Version 2.1 the arangorestore tool supports sharding. Simply point it to one of the coordinators in your cluster and it will work as usual but on sharded collections in the cluster.

If arangorestore is asked to drop and re-create a collection, it will use the same number of shards and the same shard keys as when the collection was dumped. The distribution of the shards to the servers will also be the same as at the time of the dump. This means in particular that DBservers with the same IDs as before must be present in the cluster at time of the restore.

If a collection was dumped from a single instance, one can manually add the structural description for the shard keys and the number and distribution of the shards and then the restore into a cluster will work.

If you restore a collection that was dumped from a cluster into a single ArangoDB instance, the number of shards and the shard keys will silently be ignored.

Note that in a cluster, every newly created collection will have a new ID, it is not possible to reuse the ID from the originally dumped collection. This is for safety reasons to ensure consistency of IDs.

JavaScript Modules

Introduction to JavaScript Modules

The ArangoDB uses a CommonJS compatible module and package concept. You can use the function require in order to load a module or package. It returns the exported variables and functions of the module or package.

There are some extensions to the CommonJS concept to allow ArangoDB to load Node.js modules as well.

CommonJS Modules

Unfortunately, the JavaScript libraries are just in the process of being standardized. CommonJS has defined some important modules. ArangoDB implements the following

- "console" is a well known logging facility to all the JavaScript developers. ArangoDB implements most of the Console API, with the exceptions of profile and count.

- "fs" provides a file system API for the manipulation of paths, directories, files, links, and the construction of file streams. ArangoDB implements most Filesystem/A functions.

- Modules are implemented according to Modules/1.1.1

- Packages are implemented according to Packages/1.0

ArangoDB Specific Modules

A lot of the modules, however, are ArangoDB specific. These modules are described in the following chapters.

Node Modules

ArangoDB also supports some node modules.

- "assert" implements assertion and testing functions.

- "buffer" implements a binary data type for JavaScript.

- "path" implements functions dealing with filenames and paths.

- "punycode" implements conversion functions for punycode encoding.

- "querystring" provides utilities for dealing with query strings.

- "stream" provides a streaming interface.

- "url" has utilities for URL resolution and parsing.

Bundled NPM Modules

The following NPM modules are preinstalled.

- "aqb" is the ArangoDB Query Builder and can be used to construct AQL queries with a chaining JavaScript API.

- "error-stack-parser"

- "expect.js"

- "extendible"

- "foxx_generator"

- "http-errors"

- "i"

- "joi" is a validation library that is used throughout the Foxx framework.

- "js-yaml"

- "minimatch"

- "qs" provides utilities for dealing with query strings using a different format than the querystring module.

- "ramda"

- "semver"

- "sinon"

- "underscore"

Installing NPM Modules

You can install additional modules using  npm install . Note the following limitations in ArangoDB's compatibility with node or browser modules:

- modules must be implemented in pure JavaScript (no native extensions)

- modules must be strictly synchronous (e.g. no setTimeout or promises)

- only a subset of node's built-in modules are supported (see above)

- the same limitations apply to each module's dependencies

require

require(path)

require checks if the module or package specified by path has already been loaded. If not, the content of the file is executed in a new context. Within the context you can use the global variable exports in order to export variables and functions. This variable is returned by require.

Assume that your module file is test1.js and contains

exports.func1 = function() {

print("1");

};

exports.const1 = 1;

Then you can use require to load the file and access the exports.

unix> ./arangosh

arangosh> var test1 = require("test1");

arangosh> test1.const1;

1

arangosh> test1.func1();

1

require follows the specification Modules/1.1.1.

require will inject two variables into the context of the required code:

- __filename : contains the name of the required file/module

- __dirname : contains the directory name of the required file/module

The values in  __filename  and  __dirname  can be used for generic debugging and for creating filename relative to the required file, e.g.

arangosh> var files = require("fs");

arangosh> relativeFile = files.join(__dirname, "scripts", "test.js");

/tmp/scripts/test.js

require can also be used to load JSON data. If the name of the required module ends with .json, require will run a  JSON.parse()  on the file/module contents and return the result.

Note: the purpose of require is to load modules or packages. It cannot be used to load arbitrary JavaScript files.

Modules Path versus Modules Collection

ArangoDB comes with predefined modules defined in the file-system under the path specified by startup.startup-directory. In a standard installation this point to the system share directory. Even if you are an administrator of ArangoDB you might not have write permissions to this location. On the other hand, in order to deploy some extension for ArangoDB, you might need to install additional JavaScript modules. This would require you to become root and copy the files into the share directory. In order to ease the deployment of extensions, ArangoDB uses a second mechanism to look up JavaScript modules.

JavaScript modules can either be stored in the filesystem as regular file or in the database collection _modules.

If you execute

require("com/example/extension")

then ArangoDB will try to locate the corresponding JavaScript as file as follows

- There is a cache for the results of previous require calls. First of all ArangoDB checks if com/example/extension is already in the modules cache. If it is, the export object for this module is returned. No further JavaScript is executed.

- ArangoDB will then check, if there is a file called com/example/extension.js in the system search path. If such a file exists, it is executed in a new module context and the value of exports object is returned. This value is also stored in the module cache.

- If no file can be found, ArangoDB will check if the collection _modules contains a document of the form

{

path: "/com/example/extension",

content: "...."

}

Note: The leading / is important - even if you call require without a leading /. If such a document exists, then the value of the content attribute must contain the JavaScript code of the module. This string is executed in a new module context and the value of exports object is returned. This value is also stored in the module cache.

Console Module

The implementation follows the CommonJS specification Console.

console.assert

console.assert(expression, format, argument1, ...)

Tests that an expression is true. If not, logs a message and throws an exception.

Examples

console.assert(value === "abc", "expected: value === abc, actual:", value);

console.debug

console.debug(format, argument1, ...)

Formats the arguments according to format and logs the result as debug message. Note that debug messages will only be logged if the server is started with log levels debug or trace.

String substitution patterns, which can be used in format.

- %%s string

- %%d, %%i integer

- %%f floating point number

- %%o object hyperlink

Examples

console.debug("%s", "this is a test");

console.dir

console.dir(object)

Logs a listing of all properties of the object.

Example usage:

console.dir(myObject);

console.error

console.error(format, argument1, ...)

Formats the arguments according to @FA{format} and logs the result as error message.

String substitution patterns, which can be used in format.

- %%s string

- %%d, %%i integer

- %%f floating point number

- %%o object hyperlink

Example usage:

console.error("error '%s': %s", type, message);

console.getline

console.getline()

Reads in a line from the console and returns it as string.

console.group

console.group(format, argument1, ...)

Formats the arguments according to format and logs the result as log message. Opens a nested block to indent all future messages sent. Call groupEnd to close the block. Representation of block is up to the platform, it can be an interactive block or just a set of indented sub messages.

Example usage:

console.group("user attributes");

console.log("name", user.name);

console.log("id", user.id);

console.groupEnd();

console.groupCollapsed

console.groupCollapsed(format, argument1, ...)

Same as console.group.

console.groupEnd

console.groupEnd()

Closes the most recently opened block created by a call to group.

console.info

console.info(format, argument1, ...)

Formats the arguments according to format and logs the result as info message.

String substitution patterns, which can be used in format.

- %%s string

- %%d, %%i integer

- %%f floating point number

- %%o object hyperlink

Example usage:

console.info("The %s jumped over %d fences", animal, count);

console.log

console.log(format, argument1, ...)

Formats the arguments according to format and logs the result as log message. This is an alias for console.info.

console.time

console.time(name)

Creates a new timer under the given name. Call timeEnd with the same name to stop the timer and log the time elapsed.

Example usage:

console.time("mytimer");

...

console.timeEnd("mytimer"); // this will print the elapsed time

console.timeEnd

console.timeEnd(name)

Stops a timer created by a call to time and logs the time elapsed.

console.timeEnd

console.trace()

Logs a stack trace of JavaScript execution at the point where it is called.

console.warn

console.warn(format, argument1, ...)

Formats the arguments according to format and logs the result as warn message.

String substitution patterns, which can be used in format.

- %%s string

- %%d, %%i integer

- %%f floating point number

- %%o object hyperlink

Module "fs"

The implementation tries to follow the CommonJS specification where possible. Filesystem/A/0.

Single File Directory Manipulation

exists

fs.exists(path)

Returns true if a file (of any type) or a directory exists at a given path. If the file is a broken symbolic link, returns false.

isFile

fs.isFile(path)

Returns true if the path points to a file.

isDirectory

fs.isDirectory(path)

Returns true if the path points to a directory.

size

fs.size(path)

Returns the size of the file specified by path.

mtime

fs.mtime(filename)

Returns the last modification date of the specified file. The date is returned as a Unix timestamp (number of seconds elapsed since January 1 1970).

pathSeparator

fs.pathSeparator

If you want to combine two paths you can use fs.pathSeparator instead of / or \.

join

fs.join(path, filename)

The function returns the combination of the path and filename, e.g. fs.join(Hello/World, foo.bar) would return Hello/World/foo.bar.

getTempFile

fs.getTempFile(directory, createFile)

Returns the name for a new temporary file in directory directory. If createFile is true, an empty file will be created so no other process can create a file of the same name.

Note: The directory directory must exist.

getTempPath

fs.getTempPath()

Returns the absolute path of the temporary directory

makeAbsolute

fs.makeAbsolute(path)

Returns the given string if it is an absolute path, otherwise an absolute path to the same location is returned.

chmod

fs.exists(path)

Returns true on success.

list

fs.list(path)

The functions returns the names of all the files in a directory, in lexically sorted order. Throws an exception if the directory cannot be traversed (or path is not a directory).

Note: this means that list("x") of a directory containing "a" and "b" would return ["a", "b"], not ["x/a", "x/b"].

listTree

fs.listTree(path)

The function returns an array that starts with the given path, and all of the paths relative to the given path, discovered by a depth first traversal of every directory in any visited directory, reporting but not traversing symbolic links to directories. The first path is always "", the path relative to itself.

makeDirectory

fs.makeDirectory(path)

Creates the directory specified by path.

makeDirectoryRecursive

fs.makeDirectoryRecursive(path)

Creates the directory hierarchy specified by path.

remove

fs.remove(filename)

Removes the file filename at the given path. Throws an exception if the path corresponds to anything that is not a file or a symbolic link. If "path" refers to a symbolic link, removes the symbolic link.

removeDirectory

fs.removeDirectory(path)

Removes a directory if it is empty. Throws an exception if the path is not an empty directory.

removeDirectoryRecursive

fs.removeDirectoryRecursive(path)

Removes a directory with all subelements. Throws an exception if the path is not a directory.

File IO

read

fs.read(filename)

Reads in a file and returns the content as string. Please note that the file content must be encoded in UTF-8.

read64

fs.read64(filename)

Reads in a file and returns the content as string. The file content is Base64 encoded.

readBuffer

fs.readBuffer(filename)

Reads in a file and returns its content in a Buffer object.

readFileSync

fs.readFileSync(filename, encoding)

Reads the contents of the file specified in  filename . If  encoding  is specified, the file contents will be returned as a string. Supported encodings are:

- utf8  or  utf-8

- ascii

- base64

- ucs2  or  ucs-2

- utf16le  or  utf16be

- hex

If no  encoding  is specified, the file contents will be returned in a Buffer object.

save

fs.write(filename, content)

Writes the content into a file. Content can be a string or a Buffer object.

writeFileSync

fs.writeFileSync(filename, content)

This is an alias for  fs.write(filename, content) .

Recursive Manipulation

copyRecursive

fs.copyRecursive(source, destination)

Copies source to destination. Exceptions will be thrown on:

- Failure to copy the file

- specifying a directory for destination when source is a file

- specifying a directory as source and destination

CopyFile

fs.copyFile(source, destination)

Copies source to destination. If Destination is a directory, a file of the same name will be created in that directory, else the copy will get the specified filename.

move

fs.move(source, destination)

Moves source to destination. Failure to move the file, or specifying a directory for destination when source is a file will throw an exception. Likewise, specifying a directory as source and destination will fail.

ZIP

unzipFile

fs.unzipFile(filename, outpath, skipPaths, overwrite, password)

Unzips the zip file specified by filename into the path specified by outpath. Overwrites any existing target files if overwrite is set to true.

Returns true if the file was unzipped successfully.

zipFile

fs.zipFile(filename, chdir, files, password)

Stores the files specified by files in the zip file filename. If the file filename already exists, an error is thrown. The list of input files files must be given as a list of absolute filenames. If chdir is not empty, the chdir prefix will be stripped from the filename in the zip file, so when it is unzipped filenames will be relative. Specifying a password is optional.

Returns true if the file was zipped successfully.

Module "process"

mimic parts of nodejs process module

env

env maps the systems environment to an associative array.

Module "request"

The request module provides the functionality for making HTTP requests.

require('org/arangodb/request');

Making HTTP requests

HTTP method helpers

In addition to the request function convenience shorthands are available for each HTTP method in the form of, i.e.:

- request.head(url, options)

- request.get(url, options)

- request.post(url, options)

- request.put(url, options)

- request.delete(url, options)

- request.patch(url, options)

These are equivalent to using the request function directly, i.e.:

request[method](url, options)

// is equivalent to

request({method, url, ...options});

For example:

request.get('http://localhost', {headers: {'x-session-id': 'keyboardcat'}});

// is equivalent to

request({

method: 'get',

url: 'http://localhost',

headers: {'x-session-id': 'keyboardcat'}

});

The request function

The request function can be used to make HTTP requests.

request(options)

Performs an HTTP request and returns a Response object.

Parameter

The request function takes the following options:

- url or uri: the fully-qualified URL or a parsed URL from  url.parse .

- qs (optional): object containing querystring values to be appended to the URL.

- useQuerystring: if  true , use  querystring  module to handle querystrings, otherwise use  qs  module. Default:  false .

- method (optional): HTTP method (case-insensitive). Default:  "GET" .

- headers (optional): HTTP headers (case-insensitive). Default:  {} .

- body (optional): request body. Must be a string or  Buffer , or a JSON serializable value if json is  true .

- json: if  true , body will be serialized to a JSON string and the Content-Type header will be set to  "application/json" . Additionally the response body will also be parsed as JSON (unless encoding is set to  null ). Default:  false .

- form (optional): when set to a string or object and no body has been set, body will be set to a querystring representation of that value and the Content-Type header will be set to  "application/x-www-form-urlencoded" . Also see useQuerystring.

- auth (optional): an object with the properties username and password for HTTP Basic authentication or the property bearer for HTTP Bearer token authentication.

- followRedirect: whether HTTP 3xx redirects should be followed. Default:  true .

- maxRedirects: the maximum number of redirects to follow. Default:  10 .

- encoding: encoding to be used for the response body. If set to  null , the response body will be returned as a  Buffer . Default:  "utf-8" .

- timeout: number of milliseconds to wait for a response before aborting the request.

- returnBodyOnError: whether the response body should be returned even when the server response indicates an error. Default:  true .

The function returns a Response object with the following properties:

- rawBody: the raw response body as a  Buffer .

- body: the parsed response body. If encoding was not set to  null , this is a string. If additionally json was set to  true  and the response body is well-formed JSON, this is the parsed JSON data.

- headers: an object containing the response headers. Otherwise this is identical to rawBody.

- statusCode and status: the HTTP status code of the response, e.g.  404 .

- message: the HTTP status message of the response, e.g.  Not Found .

Forms

The request module supports  application/x-www-form-urlencoded  (URL encoded) form uploads:

var res = request.post('http://service.example/upload', {form: {key: 'value'}});

// or

var res = request.post({url: 'http://service.example/upload', form: {key: 'value'}});

// or

var res = request({

method: 'post',

url: 'http://service.example/upload',

form: {key: 'value'}

});

Form data will be encoded using the qs module by default.

If you want to use the querystring module instead, simply use the useQuerystring option.

JSON

If you want to submit JSON-serializable values as request bodies, just set the json option:

var res = request.post('http://service.example/notify', {body: {key: 'value'}, json: true});

// or

var res = request.post({url: 'http://service.example/notify', body: {key: 'value'}, json: true});

// or

var res = request({

method: 'post',

url: 'http://service.example/notify',

body: {key: 'value'},

json: true

});

HTTP authentication

The request module supports both HTTP Basic authentication. Just pass the credentials via the auth option:

var res = request.get(

'http://service.example/secret',

{auth: {username: 'jcd', password: 'bionicman'}}

);

// or

var res = request.get({

url: 'http://service.example/secret',

auth: {username: 'jcd', password: 'bionicman'}

});

// or

var res = request({

method: 'get',

url: 'http://service.example/secret',

auth: {username: 'jcd', password: 'bionicman'}

});

Alternatively you can supply the credentials via the URL:

var username = 'jcd';

var password = 'bionicman';

var res = request.get(

'http://' +

encodeURIComponent(username) +

':' +

encodeURIComponent(password) +

'@service.example/secret'

);

You can also use Bearer token authentication:

var res = request.get(

'http://service.example/secret',

{auth: {bearer: 'keyboardcat'}}

);

// or

var res = request.get({

url: 'http://service.example/secret',

auth: {bearer: 'keyboardcat'}

});

// or

var res = request({

method: 'get',

url: 'http://service.example/secret',

auth: {bearer: 'keyboardcat'}

});

Module "actions"

The action module provides the infrastructure for defining HTTP actions.

Basics

Error message

actions.getErrorMessage(code)

Returns the error message for an error code.

Standard HTTP Result Generators

actions.defineHttp(options)

Defines a new action. The options are as follows:

options.url

The URL, which can be used to access the action. This path might contain slashes. Note that this action will also be called, if a url is given such that options.url is a prefix of the given url and no longer definition matches.

options.prefix

If false, then only use the action for exact matches. The default is true.

options.callback(request, response)

The request argument contains a description of the request. A request parameter foo is accessible as request.parametrs.foo. A request header bar is accessible as request.headers.bar. Assume that the action is defined for the url /foo/bar and the request url is /foo/bar/hugo/egon. Then the suffix parts [ "hugo", "egon" ] are availible in request.suffix.

The callback must define fill the response.

- response.responseCode: the response code

- response.contentType: the content type of the response

- response.body: the body of the response

You can use the functions ResultOk and ResultError to easily generate a response.

Result ok

actions.resultOk(req, res, code, result, headers)

The function defines a response. code is the status code to return. result is the result object, which will be returned as JSON object in the body. headers is an array of headers to returned. The function adds the attribute error with value false and code with value code to the result.

Result bad

actions.resultBad(req, res, error-code, msg, headers)

The function generates an error response.

Result not found

actions.resultNotFound(req, res, code, msg, headers)

The function generates an error response.

Result unsupported

actions.resultUnsupported(req, res, headers)

The function generates an error response.

Result error

actions.resultError(req, res, code, errorNum, errorMessage, headers, keyvals)*

The function generates an error response. The response body is an array with an attribute errorMessage containing the error message errorMessage, error containing true, code containing code, errorNum containing errorNum, and errorMessage containing the error message errorMessage. keyvals are mixed into the result.

Result not Implemented

actions.resultNotImplemented(req, res, msg, headers)

The function generates an error response.

Result permanent redirect

actions.resultPermanentRedirect(req, res, options, headers)

The function generates a redirect response.

Result temporary redirect

actions.resultTemporaryRedirect(req, res, options, headers)

The function generates a redirect response.

ArangoDB Result Generators

Collection not found

actions.collectionNotFound(req, res, collection, headers)

The function generates an error response.

Index not found

actions.indexNotFound(req, res, collection, index, headers)

The function generates an error response.

Result exception

actions.resultException(req, res, err, headers, verbose)

The function generates an error response. If @FA{verbose} is set to true or not specified (the default), then the error stack trace will be included in the error message if available. If @FA{verbose} is a string it will be prepended before the error message and the stacktrace will also be included.

Module "queries"

The query module provides the infrastructure for working with currently running AQL queries via arangosh.

Properties

queries.properties()  Returns the servers current query tracking configuration; we change the slow query threshold to get better results:

arangosh> var queries = require("org/arangodb/aql/queries");

arangosh> queries.properties();

arangosh> queries.properties({slowQueryThreshold: 1});

show execution results

Currently running queries

We create a task that spawns queries, so we have nice output. Since this task uses resources, you may want to increase  period  (and not forget to remove it... afterwards):

arangosh> var theQuery = 'FOR sleepLoooong IN 1..5 LET sleepLoooonger = SLEEP(1000) RETURN sleepLoooong';

arangosh> var tasks = require("org/arangodb/tasks");

arangosh> tasks.register({

........>  id: "mytask-1",

........>  name: "this is a sample task to spawn a slow aql query",

........>  command: "require('internal').db._query('" + theQuery + "');"

........> });

arangosh> queries.current();

show execution results

The function returns the currently running AQL queries as an array.

Slow queries

The function returns the last AQL queries that exceeded the slow query threshold as an array:

arangosh> queries.slow();

[ ]

Clear slow queries

Clear the list of slow AQL queries:

arangosh> queries.clearSlow();

{

"code" : 200

}

arangosh> queries.slow();

[ ]

Kill

Kill a running AQL query:

arangosh> var runningQueries = queries.current().filter(function(query) {

........>   return query.query === theQuery;

........> });

arangosh> queries.kill(runningQueries[0].id);

{

"code" : 200

}

Module "planner"

Cluster Module

This module contains functions to plan, launch and shutdown clusters of ArangoDB instances. We distinguish between the planning phase of a cluster and the startup phase. The planning involves determining how many processes in which role to run on which server, what ports and endpoints to use, as well as the sequence of events to startup the whole machinery. The result of such a planning phase is a "cluster plan". This in turn can be given to a "Kickstarter", which uses the plan to actually fire up the necessary processes. This is done via "dispatchers". A dispatcher is nothing but a regular ArangoDB instance, compiled with the cluster extensions, but not running in cluster mode. It exposes a REST API to help in the planning and running of a cluster, mainly by starting up further processes on the local machine. The planner needs a complete description of all dispatchers in the system, which basically describes the set of machines used for the cluster. These dispatchers are also used during the planning to find free network ports.

Here are the details of the functionality:

Require

new require("org/arangodb/cluster").Planner(userConfig)

This constructor builds a cluster planner object. The one and only argument is an object that can have the properties described below. The planner can plan clusters on a single machine (basically for testing purposes) and on multiple machines. The resulting "cluster plans" can be used by the kickstarter to start up the processes comprising the cluster, including the agency. To this end, there has to be one dispatcher on every machine participating in the cluster. A dispatcher is a simple instance of ArangoDB, compiled with the cluster extensions, but not running in cluster mode. This is why the configuration option dispatchers below is of central importance.

- dispatchers: an object with a property for each dispatcher, the property name is the ID of the dispatcher and the value should be an object with at least the property endpoint containing the endpoint of the corresponding dispatcher. Further optional properties are:

- avoidPorts which is an object

in which all port numbers that should not be used are bound to

true, default is empty, that is, all ports can be used

- arangodExtraArgs, which is a list of additional

command line arguments that will be given to DBservers and

coordinators started by this dispatcher, the default is

an empty list. These arguments will be appended to those

produced automatically, such that one can overwrite

things with this.

- allowCoordinators, which is a boolean value indicating

whether or not coordinators should be started on this

dispatcher, the default is *true*

- allowDBservers, which is a boolean value indicating

whether or not DBservers should be started on this dispatcher,

the default is *true*

- allowAgents, which is a boolean value indicating whether or

not agents should be started on this dispatcher, the default is

true

- username, which is a string that contains the user name

for authentication with this dispatcher

- passwd, which is a string that contains the password

for authentication with this dispatcher, if not both

username and passwd are set, then no authentication

is used between dispatchers. Note that this will not work

if the dispatchers are configured with authentication.

If .dispatchers is empty (no property), then an entry for the local arangod itself is automatically added. Note that if the only configured dispatcher has endpoint tcp://localhost:, all processes are started in a special "local" mode and are configured to bind their endpoints only to the localhost device. In all other cases both agents and arangod instances bind their endpoints to all available network devices.

- numberOfAgents: the number of agents in the agency, usually there is no reason to deviate from the default of 3. The planner distributes them amongst the dispatchers, if possible.

- agencyPrefix: a string that is used as prefix for all keys of configuration data stored in the agency.

- numberOfDBservers: the number of DBservers in the cluster. The planner distributes them evenly amongst the dispatchers.

- startSecondaries: a boolean flag indicating whether or not secondary servers are started. In this version, this flag is silently ignored, since we do not yet have secondary servers.

- numberOfCoordinators: the number of coordinators in the cluster, the planner distributes them evenly amongst the dispatchers.

- DBserverIDs: a list of DBserver IDs (strings). If the planner runs out of IDs it creates its own ones using DBserver concatenated with a unique number.

- coordinatorIDs: a list of coordinator IDs (strings). If the planner runs out of IDs it creates its own ones using Coordinator concatenated with a unique number.

- dataPath: this is a string and describes the path under which the agents, the DBservers and the coordinators store their data directories. This can either be an absolute path (in which case all machines in the clusters must use the same path), or it can be a relative path. In the latter case it is relative to the directory that is configured in the dispatcher with the cluster.data-path option (command line or configuration file). The directories created will be called data-PREFIX-ID where PREFIX is replaced with the agency prefix (see above) and ID is the ID of the DBserver or coordinator.

- logPath: this is a string and describes the path under which the DBservers and the coordinators store their log file. This can either be an absolute path (in which case all machines in the cluster must use the same path), or it can be a relative path. In the latter case it is relative to the directory that is configured in the dispatcher with the cluster.log-path option.

- arangodPath: this is a string and describes the path to the actual executable arangod that will be started for the DBservers and coordinators. If this is an absolute path, it obviously has to be the same on all machines in the cluster as described for dataPath. If it is an empty string, the dispatcher uses the executable that is configured with the cluster.arangod-path option, which is by default the same executable as the dispatcher uses.

- agentPath: this is a string and describes the path to the actual executable that will be started for the agents in the agency. If this is an absolute path, it obviously has to be the same on all machines in the cluster, as described for arangodPath. If it is an empty string, the dispatcher uses its cluster.agent-path option.

- agentExtPorts: a list of port numbers to use for the external ports of the agents. When running out of numbers in this list, the planner increments the last one used by one for every port needed. Note that the planner checks availability of the ports during the planning phase by contacting the dispatchers on the different machines, and uses only ports that are free during the planning phase. Obviously, if those ports are connected before the actual startup, things can go wrong.

- agentIntPorts: a list of port numbers to use for the internal ports of the agents. The same comments as for agentExtPorts apply.

- DBserverPorts: a list of port numbers to use for the DBservers. The same comments as for agentExtPorts apply.

- coordinatorPorts: a list of port numbers to use for the coordinators. The same comments as for agentExtPorts apply.

- useSSLonDBservers: a boolean flag indicating whether or not we use SSL on all DBservers in the cluster

- useSSLonCoordinators: a boolean flag indicating whether or not we use SSL on all coordinators in the cluster

- valgrind: a string to contain the path of the valgrind binary if we should run the cluster components in it

- valgrindopts: commandline options to the valgrind process

- valgrindXmlFileBase: pattern for logfiles

- valgrindTestname: name of test to add to the logfiles

- valgrindHosts: which host classes should run in valgrind? Coordinator / DBServer

All these values have default values. Here is the current set of default values:

{

"agencyPrefix"            : "arango",

"numberOfAgents"          : 1,

"numberOfDBservers"       : 2,

"startSecondaries"        : false,

"numberOfCoordinators"    : 1,

"DBserverIDs"             : ["Pavel", "Perry", "Pancho", "Paul", "Pierre",

"Pit", "Pia", "Pablo" ],

"coordinatorIDs"          : ["Claus", "Chantalle", "Claire", "Claudia",

"Claas", "Clemens", "Chris" ],

"dataPath"                : "",   // means configured in dispatcher

"logPath"                 : "",   // means configured in dispatcher

"arangodPath"             : "",   // means configured as dispatcher

"agentPath"               : "",   // means configured in dispatcher

"agentExtPorts"           : [4001],

"agentIntPorts"           : [7001],

"DBserverPorts"           : [8629],

"coordinatorPorts"        : [8530],

"dispatchers"             : {"me": {"endpoint": "tcp://localhost:"}},

// this means only we as a local instance

"useSSLonDBservers"       : false,

"useSSLonCoordinators"    : false

};

Get Plan

Planner.getPlan()

returns the cluster plan as a JavaScript object. The result of this method can be given to the constructor of a Kickstarter.

Require

new require("org/arangodb/cluster").Kickstarter(plan)

This constructor constructs a kickstarter object. Its first argument is a cluster plan as for example provided by the planner (see Cluster Planner Constructor and the general explanations before this reference). The second argument is optional and is taken to be "me" if omitted, it is the ID of the dispatcher this object should consider itself to be. If the plan contains startup commands for the dispatcher with this ID, these commands are executed immediately. Otherwise they are handed over to another responsible dispatcher via a REST call.

The resulting object of this constructors allows to launch, shutdown, relaunch the cluster described in the plan.

Launch

Kickstarter.launch()

This starts up a cluster as described in the plan which was given to the constructor. To this end, other dispatchers are contacted as necessary. All startup commands for the local dispatcher are executed immediately.

The result is an object that contains information about the started processes, this object is also stored in the Kickstarter object itself. We do not go into details here about the data structure, but the most important information are the process IDs of the started processes. The corresponding see shutdown method needs this information to shut down all processes.

Note that all data in the DBservers and all log files and all agency information in the cluster is deleted by this call. This is because it is intended to set up a cluster for the first time. See the relaunch method for restarting a cluster without data loss.

Check Cluster Health

Kickstarter.isHealthy()

This checks that all processes belonging to a running cluster are healthy. To this end, other dispatchers are contacted as necessary. At this stage it is only checked that the processes are still up and running.

Shutdown

Kickstarter.shutdown()

This shuts down a cluster as described in the plan which was given to the constructor. To this end, other dispatchers are contacted as necessary. All processes in the cluster are gracefully shut down in the right order.

Relaunch

Kickstarter.relaunch()

This starts up a cluster as described in the plan which was given to the constructor. To this end, other dispatchers are contacted as necessary. All startup commands for the local dispatcher are executed immediately.

The result is an object that contains information about the started processes, this object is also stored in the Kickstarter object itself. We do not go into details here about the data structure, but the most important information are the process IDs of the started processes. The corresponding shutdown method needs this information to shut down all processes.

Note that this methods needs that all data in the DBservers and the agency information in the cluster are already set up properly. See the launch method for starting a cluster for the first time.

Upgrade

Kickstarter.upgrade(username, passwd)

This performs an upgrade procedure on a cluster as described in the plan which was given to the constructor. To this end, other dispatchers are contacted as necessary. All commands for the local dispatcher are executed immediately. The basic approach for the upgrade is as follows: The agency is started first (exactly as in relaunch), no configuration is sent there (exactly as in the relaunch action), all servers are first started with the option "--upgrade" and then normally. In the end, the upgrade-database.js script is run on one of the coordinators, as in the launch action.

The result is an object that contains information about the started processes, this object is also stored in the Kickstarter object itself. We do not go into details here about the data structure, but the most important information are the process IDs of the started processes. The corresponding shutdown method needs this information to shut down all processes.

Note that this methods needs that all data in the DBservers and the agency information in the cluster are already set up properly. See the launch method for starting a cluster for the first time.

Cleanup

Kickstarter.cleanup()

This cleans up all the data and logs of a previously shut down cluster. To this end, other dispatchers are contacted as necessary. Use shutdown first and use with caution, since potentially a lot of data is being erased with this call!

Write-ahead log

This module provides functionality for administering the write-ahead logs.

Configuration

internal.wal.properties()

Retrieves the configuration of the write-ahead log. The result is a JSON array with the following attributes:

- allowOversizeEntries: whether or not operations that are bigger than a single logfile can be executed and stored

- logfileSize: the size of each write-ahead logfile

- historicLogfiles: the maximum number of historic logfiles to keep

- reserveLogfiles: the maximum number of reserve logfiles that ArangoDB allocates in the background

- syncInterval: the interval for automatic synchronization of not-yet synchronized write-ahead log data (in milliseconds)

- throttleWait: the maximum wait time that operations will wait before they get aborted if case of write-throttling (in milliseconds)

- throttleWhenPending: the number of unprocessed garbage-collection operations that, when reached, will activate write-throttling. A value of 0 means that write-throttling will not be triggered.

Examples

arangosh> require("internal").wal.properties();

show execution results

internal.wal.properties(properties)

Configures the behavior of the write-ahead log. properties must be a JSON JSON object with the following attributes:

- allowOversizeEntries: whether or not operations that are bigger than a single logfile can be executed and stored

- logfileSize: the size of each write-ahead logfile

- historicLogfiles: the maximum number of historic logfiles to keep

- reserveLogfiles: the maximum number of reserve logfiles that ArangoDB allocates in the background

- throttleWait: the maximum wait time that operations will wait before they get aborted if case of write-throttling (in milliseconds)

- throttleWhenPending: the number of unprocessed garbage-collection operations that, when reached, will activate write-throttling. A value of 0 means that write-throttling will not be triggered.

Specifying any of the above attributes is optional. Not specified attributes will be ignored and the configuration for them will not be modified.

Examples

arangosh> require("internal").wal.properties({ allowOverSizeEntries: true, logfileSize: 32 * 1024 * 1024 });

show execution results

Flushing

internal.wal.flush(waitForSync, waitForCollector)

Flushes the write-ahead log. By flushing the currently active write-ahead logfile, the data in it can be transferred to collection journals and datafiles. This is useful to ensure that all data for a collection is present in the collection journals and datafiles, for example, when dumping the data of a collection.

The waitForSync option determines whether or not the operation should block until the not-yet synchronized data in the write-ahead log was synchronized to disk.

The waitForCollector operation can be used to specify that the operation should block until the data in the flushed log has been collected by the write-ahead log garbage collector. Note that setting this option to true might block for a long time if there are long-running transactions and the write-ahead log garbage collector cannot finish garbage collection.

Examples

arangosh> require("internal").wal.flush();

Task Management

Introduction to Task Management in ArangoDB

ArangoDB can execute user-defined JavaScript functions as one-shot or periodic tasks. This functionality can be used to implement timed or recurring jobs in the database.

Tasks in ArangoDB consist of a JavaScript snippet or function that is executed when the task is scheduled. A task can be a one-shot task (meaning it is run once and not repeated) or a periodic task (meaning that it is re-scheduled after each execution). Tasks can have optional parameters, which are defined at task setup time. The parameters specified at task setup time will be passed as arguments to the task whenever it gets executed. Periodic Tasks have an execution frequency that needs to be specified when the task is set up. One-shot tasks have a configurable delay after which they'll get executed.

Tasks will be executed on the server they have been set up on. Tasks will not be shipped around in a cluster. A task will be executed in the context of the database it was created in. However, when dropping a database, any tasks that were created in the context of this database will remain active. It is therefore sensible to first unregister all active tasks for a database before dropping the database.

Tasks registered in ArangoDB will be executed until the server gets shut down or restarted. After a restart of the server, any user-defined one-shot or periodic tasks will be lost.

Commands for Working with Tasks

ArangoDB provides the following commands for working with tasks. All commands can be accessed via the tasks module, which can be loaded like this:

require("org/arangodb/tasks");

Please note that the tasks module is available inside the ArangoDB server only. It cannot be used from the ArangoShell or ArangoDB's web interface.

Register a task

To register a task, the JavaScript snippet or function needs to be specified in addition to the execution frequency. Optionally, a task can have an id and a name. If no id is specified, it will be auto-assigned for a new task. The task id is also the means to access or unregister a task later. Task names are informational only. They can be used to make a task distinguishable from other tasks also running on the server.

The following server-side commands register a task. The command to be executed is a JavaScript string snippet which prints a message to the server's logfile:

var tasks = require("org/arangodb/tasks");

tasks.register({

id: "mytask-1",

name: "this is a snippet task",

period: 15,

command: "require('console').log('hello from snippet task');"

});

The above has register a task with id mytask-1, which will be executed every 15 seconds on the server. The task will write a log message whenever it is invoked.

Tasks can also be set up using a JavaScript callback function like this:

var tasks = require("org/arangodb/tasks");

tasks.register({

id: "mytask-2",

name: "this is a function task",

period: 15,

command: function () {

require('console').log('hello from function task');

}

});

It is important to note that the callback function is late bound and will be executed in a different context than in the creation context. The callback function must therefore not access any variables defined outside of its own scope. The callback function can still define and use its own variables.

To pass parameters to a task, the params attribute can be set when registering a task. Note that the parameters are limited to data types usable in JSON (meaning no callback functions can be passed as parameters into a task):

var tasks = require("org/arangodb/tasks");

tasks.register({

id: "mytask-3",

name: "this is a parameter task",

period: 15,

command: function (params) {

var greeting = params.greeting;

var data = JSON.stringify(params.data);

require('console').log('%s from parameter task: %s', greeting, data);

},

params: { greeting: "hi", data: "how are you?" }

});

Registering a one-shot task works the same way, except that the period attribute must be omitted. If period is omitted, then the task will be executed just once. The task invocation delay can optionally be specified with the offset attribute:

var tasks = require("org/arangodb/tasks");

tasks.register({

id: "mytask-once",

name: "this is a one-shot task",

offset: 10,

command: function (params) {

require('console').log('you will see me just once!');

}

});

Note: When specifying an offset value of 0, ArangoDB will internally add a very small value to the offset so will be slightly greater than zero.

Note: Normally  offset  cannot be greater than  interval . The only way to set  offset  greater than  interval  is to register a non-periodic timer with the specified offset and when this fires this can install a periodic timer with an offset of 0.

Unregister a task

After a task has been registered, it can be unregistered using its id:

var tasks = require("org/arangodb/tasks");

tasks.unregister("mytask-1");

Note that unregistering a non-existing task will throw an exception.

List all tasks

To get an overview of which tasks are registered, there is the get method. If the get method is called without any arguments, it will return an array of all tasks:

var tasks = require("org/arangodb/tasks");

tasks.get();

If get is called with a task id argument, it will return information about this particular task:

var tasks = require("org/arangodb/tasks");

tasks.get("mytask-3");

The created attribute of a task reveals when a task was created. It is returned as a Unix timestamp.

Using jsUnity and node-jscoverage

jsUnity

The ArangoDB contains a wrapper for jsUnity, a lightweight universal JavaScript unit testing framework.

Running jsUnity Tests

Assume that you have a test file containing

function exampleTestSuite () {

return {

testSizeOfTestCollection : function () {

assertEqual(5, 5);

};

}

jsUnity.run(aqlTestSuite);

return jsunity.done();

Then you can run the test suite using jsunity.runTest

arangosh> require("jsunity").runTest("test.js");

2012-01-28T19:10:23Z [10671] INFO Running aqlTestSuite

2012-01-28T19:10:23Z [10671] INFO 1 test found

2012-01-28T19:10:23Z [10671] INFO [PASSED] testSizeOfTestCollection

2012-01-28T19:10:23Z [10671] INFO 1 test passed

2012-01-28T19:10:23Z [10671] INFO 0 tests failed

2012-01-28T19:10:23Z [10671] INFO 1 millisecond elapsed

Administrating ArangoDB

AppendOnly/MVCC

Instead of overwriting existing documents, ArangoDB will create a new version of modified documents. This is even the case when a document gets deleted. The two benefits are:

- Objects can be stored coherently and compactly in the main memory.

- Objects are preserved, isolated writing and reading transactions allow accessing these objects for parallel operations.

The system collects obsolete versions as garbage, recognizing them as forsaken. Garbage collection is asynchronous and runs parallel to other processes.

Mostly Memory/Durability

Database documents are stored in memory-mapped files. Per default, these memory-mapped files are synced regularly but not instantly. This is often a good tradeoff between storage performance and durability. If this level of durability is too low for an application, the server can also sync all modifications to disk instantly. This will give full durability but will come with a performance penalty as each data modification will trigger a sync I/O operation.

Durability Configuration

Global Configuration

There are global configuration values for durability, which can be adjusted by specifying the following configuration options:

--database.wait-for-sync boolean

Default wait-for-sync value. Can be overwritten when creating a new collection.

The default is false.

--database.force-sync-properties boolean

Force syncing of collection properties to disk after creating a collection or updating its properties.

If turned off, no fsync will happen for the collection and database properties stored in  parameter.json  files in the file system. Turning off this option will speed up workloads that create and drop a lot of collections (e.g. test suites).

The default is true.

--wal.sync-interval

The interval (in milliseconds) that ArangoDB will use to automatically synchronize data in its write-ahead logs to disk. Automatic syncs will only be performed for not-yet synchronized data, and only for operations that have been executed without the waitForSync attribute.

Per-collection configuration

You can also configure the durability behavior on a per-collection basis. Use the ArangoDB shell to change these properties.

collection.properties()

Returns an object containing all collection properties.

- waitForSync: If true creating a document will only return after the data was synced to disk.

- journalSize : The size of the journal in bytes.

- isVolatile: If true then the collection data will be kept in memory only and ArangoDB will not write or sync the data to disk.

- keyOptions (optional) additional options for key generation. This is a JSON array containing the following attributes (note: some of the attributes are optional):

- type: the type of the key generator used for the collection.

- allowUserKeys: if set to true, then it is allowed to supply own key values in the _key attribute of a document. If set to false, then the key generator will solely be responsible for generating keys and supplying own key values in the _key attribute of documents is considered an error.

- increment: increment value for autoincrement key generator. Not used for other key generator types.

- offset: initial offset value for autoincrement key generator. Not used for other key generator types.

- indexBuckets: number of buckets into which indexes using a hash table are split. The default is 16 and this number has to be a power of 2 and less than or equal to 1024.

For very large collections one should increase this to avoid long pauses when the hash table has to be initially built or resized, since buckets are resized individually and can be initially built in parallel. For example, 64 might be a sensible value for a collection with 100 000 000 documents. Currently, only the edge index respects this value, but other index types might follow in future ArangoDB versions. Changes (see below) are applied when the collection is loaded the next time.

In a cluster setup, the result will also contain the following attributes:

- numberOfShards: the number of shards of the collection.

- shardKeys: contains the names of document attributes that are used to determine the target shard for documents.

collection.properties(properties)

Changes the collection properties. properties must be a object with one or more of the following attribute(s):

- waitForSync: If true creating a document will only return after the data was synced to disk.

- journalSize : The size of the journal in bytes.

- indexBuckets : See above, changes are only applied when the collection is loaded the next time.

Note: it is not possible to change the journal size after the journal or datafile has been created. Changing this parameter will only effect newly created journals. Also note that you cannot lower the journal size to less then size of the largest document already stored in the collection.

Note: some other collection properties, such as type, isVolatile, or keyOptions cannot be changed once the collection is created.

Examples

Read all properties

arangosh> db.example.properties();

show execution results

Change a property

arangosh> db.example.properties({ waitForSync : true });

show execution results

Per-operation configuration

Many data-modification operations and also ArangoDB's transactions allow to specify a waitForSync attribute, which when set ensures the operation data has been synchronized to disk when the operation returns.

Disk-Usage Configuration

The amount of disk space used by ArangoDB is determined by a few configuration options.

Global Configuration

The total amount of disk storage required by ArangoDB is determined by the size of the write-ahead logfiles plus the sizes of the collection journals and datafiles.

There are the following options for configuring the number and sizes of the write-ahead logfiles:

--wal.reserve-logfiles

The maximum number of reserve logfiles that ArangoDB will create in a background process. Reserve logfiles are useful in the situation when an operation needs to be written to a logfile but the reserve space in the logfile is too low for storing the operation. In this case, a new logfile needs to be created to store the operation. Creating new logfiles is normally slow, so ArangoDB will try to pre-create logfiles in a background process so there are always reserve logfiles when the active logfile gets full. The number of reserve logfiles that ArangoDB keeps in the background is configurable with this option.

--wal.historic-logfiles

The maximum number of historic logfiles that ArangoDB will keep after they have been garbage-collected. If no replication is used, there is no need to keep historic logfiles except for having a local changelog.

In a replication setup, the number of historic logfiles affects the amount of data a slave can fetch from the master's logs. The more historic logfiles, the more historic data is available for a slave, which is useful if the connection between master and slave is unstable or slow. Not having enough historic logfiles available might lead to logfile data being deleted on the master already before a slave has fetched it.

--wal.logfile-size

Specifies the filesize (in bytes) for each write-ahead logfile. The logfile size should be chosen so that each logfile can store a considerable amount of documents. The bigger the logfile size is chosen, the longer it will take to fill up a single logfile, which also influences the delay until the data in a logfile will be garbage-collected and written to collection journals and datafiles. It also affects how long logfile recovery will take at server start.

--wal.allow-oversize-entries

Whether or not it is allowed to store individual documents that are bigger than would fit into a single logfile. Setting the option to false will make such operations fail with an error. Setting the option to true will make such operations succeed, but with a high potential performance impact. The reason is that for each oversize operation, an individual oversize logfile needs to be created which may also block other operations. The option should be set to false if it is certain that documents will always have a size smaller than a single logfile.

--wal.suppress-shape-information

Setting this variable to true will lead to no shape information being written into the write-ahead logfiles for documents or edges. While this is a good optimization for a single server to save memory (and disk space), it it will effectively disable using the write-ahead log as a reliable source for replicating changes to other servers. A master server with this option set to true will not be able to fully reproduce the structure of saved documents after a collection has been deleted. In case a replication client requests a document for which the collection is already deleted, the master will return an empty document. Note that this only affects replication and not normal operation on the master.

Do not set this variable to true on a server that you plan to use as a replication master When data gets copied from the write-ahead logfiles into the journals or datafiles of collections, files will be created on the collection level. How big these files are is determined by the following global configuration value:

--database.maximal-journal-size size

Maximal size of journal in bytes. Can be overwritten when creating a new collection. Note that this also limits the maximal size of a single document.

The default is 32MB.

Per-collection configuration

The journal size can also be adjusted on a per-collection level using the collection's properties method.

More advanced ArangoDB topics

This chapter covers technical details of the database system and how it can be benchmarked.

Arangob

Arangob is ArangoDB's benchmark and test tool. It can be used to issue test requests to the database for performance and server function testing. It supports parallel querying and batch requests.

Related blog posts:

- Measuring ArangoDB insert performance

- Gain factor of 5 using batch requests

Startup options

- --async: Send asynchronous requests. The default value is false.

- --batch-size: Number of operations to send per batch. Use 0 to disable batching (this is the default).

- --collection: Name of collection to use in test (only relevant for tests that invoke collections).

- --complexity: Complexity value for test case (default: 1). Meaning depends on test case.

- --concurrency: Number of parallel threads that will issue requests (default: 1).

- --configuration: Read configuration from file.

- --delay: Use a startup delay. This is only necessary when run in series. The default value is false.

- --keep-alive: Use HTTP keep-alive (default: true).

- --progress: Show progress of benchmark, on every 20th request. Set to false to disable intermediate logging. The default value is true.

- --requests: Total number of requests to perform (default: 1000).

- --server.endpoint: Server endpoint to connect to, consisting of protocol, IP address and port. Defaults to tcp://localhost:8529.

- --server.database: Database name to use when connecting (default: "_system").

- --server.username: Username to use when connecting (default: "root").

- --server.password: Password to use when connecting. Don't specify this option to get a password prompt.

- --server.disable-authentication: Disable the password prompt and authentication when connecting to the server.

- --test-case: Name of test case to perform (default: "version"). Possible values:

- version

- document

- collection,

- import-document

- hash

- skiplist

- edge

- shapes

- shapes-append

- random-shapes

- crud

- crud-append

- crud-write-read

- aqltrx

- counttrx

- multitrx

- multi-collection

- aqlinsert

- aqlv8

- --verbose: Print out replies if the HTTP header indicates DB errors. (default: false).

Examples

arangob

Starts Arangob with the default user and server endpoint.

--test-case version --requests 1000 --concurrency 1

Runs the 'version' test case with 1000 requests, without concurrency.

--test-case document --requests 1000 --concurrency 2

Runs the 'document' test case with 2000 requests, with two concurrent threads.

--test-case document --requests 1000 --concurrency 2 --async true

Runs the 'document' test case with 2000 requests, with concurrency 2, with async requests.

--test-case document --requests 1000 --concurrency 2 --batch-size 10

Runs the 'document' test case with 2000 requests, with concurrency 2, using batch requests.

Write-ahead log

Starting with version 2.2 ArangoDB stores all data-modification operation in its write-ahead log. The write-ahead log is sequence of append-only files containing all the write operations that were executed on the server.

It is used to run data recovery after a server crash, and can also be used in a replication setup when slaves need to replay the same sequence of operations as on the master.

By default, each write-ahead logfile is 32 MB big. This size is configurable via the option --wal.logfile-size.

When a write-ahead logfile is full, it is set to read-only, and following operations will be written into the next write-ahead logfile. By default, ArangoDB will reserve some spare logfiles in the background so switching logfiles should be fast. How many reserve logfiles ArangoDB will try to keep available in the background can be controlled by the configuration option --wal.reserve-logfiles.

Data contained in full datafiles will eventually be transferred into the journals or datafiles of collections. Only the "surviving" documents will be copied over. When all remaining operations from a write-ahead logfile have been copied over into the journals or datafiles of the collections, the write-ahead logfile can safely be removed if it is not used for replication.

Long-running transactions prevent write-ahead logfiles from being fully garbage-collected because it is unclear whether a transaction will commit or abort. Long-running transactions can thus block the garbage-collection progress and should therefore be avoided at all costs.

On a system that acts as a replication master, it is useful to keep a few of the already collected write-ahead logfiles so replication slaves still can fetch data from them if required. How many collected logfiles will be kept before they get deleted is configurable via the option --wal.historic-logfiles.

For all write-ahead log configuration options, please refer to the page Write-ahead log options.

Serverside db-Object implementation

We already talked about the arangosh db Object implementation, Now a little more about the server version, so the following examples won't work properly in arangosh.

Server-side methods of the db object will return an  [object ShapedJson] . This datatype is a very lightweight JavaScript object that contains an internal pointer to where the document data are actually stored in memory or on disk. Especially this is not a fullblown copy of the document's complete data.

When such an object's property is accessed, this will invoke an accessor function. For example, accessing  doc.name  of such an object will call a C++ function behind the scenes to fetch the actual value for the property  name . When a property is written to this, it will also trigger an accessor function. This accessor function will first copy all property values into the object's own properties, and then discard the pointer to the data in memory. From this point on, the accessor functions will not do anything special, so the object will behave like a normal JavaScript object.

All of this is done for performance reasons. It often allows ommitting the creation of big JavaScript objects that contain lots of data. For example, if all thats needed from a document is a single property, fully constructing the document as a JavaScript object has a high overhead (CPU time for processing, memory, plus later V8 garbage collection).

Here's an example:

var ShapedJson = require("org/arangodb").ShapedJson;

// fetches document from collection into a JavaScript object

var doc = db.test.any();

// check if the document object is a shaped object

// returns true for shaped objects, false otherwise

doc instanceof ShapedJson;

// invokes the property read accessor. returns property value byValue

doc.name;

// invokes the property write accessor. will copy document data

// into the JavaScript object once

// and store the value in the property as requested

doc.name = "test";

// doc will now behave like a regular object

doc.foo = "bar";

There is one gotcha though with such objects: the accessor functions are only invoked when accessing top level properties. When accessing nested properties, the accessor will only be called for the top level property, which will then return the requested property's value. Accessing a subproperty of this returned property however does not have a reference to the original object.

Here's an example:

// create an object with a nested property

db.test.save({ name: { first: "Jan" } });

doc;

{

"_id" : "test/2056013422404",

"_rev" : "2056013422404",

"_key" : "2056013422404",

"name" : {

"first" : "Jan"

}

}

// now modify the nested property

doc.name.first = "test";

doc;

{

"_id" : "test/2056013422404",

"_rev" : "2056013422404",

"_key" : "2056013422404",

"name" : {

"first" : "Jan"  /* oops */

}

}

So what happened here? The statement  doc.name.first = "test"  calls the read accessor for property  name  first. This produces an object  {"name":"Jan"}  whose property  first  is modifed directly afterwards. But the object  {"name":"Jan"}  is a temporary object and not the same ( === ) object as  doc.name . This is why updating the nested property effectively failed.

There is no way to detect this in a read accessor unfortunately. It does not have any idea about what will be done with the returned object. So this case cannot be tracked/trapped in the accessor.

A workaround for this problem would be to clone the object in the user code if the document is going to be modified. This will make all modification safe. The cloning can also be made conditional for cases when the object is an instance of  ShapedJson  or when nested properties are to be accessed. Cloning is not required when the object is no  ShapedJson  or when only top level properties are accessed.

Only those documents that are stored in a collections datafile will be returned as  ShapedJson . The documents, that are still in the write-ahead-log, will always be returned as regular JavaScript objects, as they are not yet shaped and the optimization would not work. However, when a document is transfered from the write-ahead-log to the collection's datafile cannot be safely predicted by an application, so the same document can be returned in one way or the other. The only safe way is to check if the object is an instance of  ShapedJson  as above.

Datafile Debugger

In Case Of Disaster

AranagoDB uses append-only journals. Data corruption should only occur when the database server is killed. In this case, the corruption should only occur in the last object(s) that have being written to the journal.

If a corruption occurs within a normal datafile, then this can only happen if a hardware fault occurred.

If a journal or datafile is corrupt, shut down the database server and start the program

unix> arango-dfdb

in order to check the consistency of the datafiles and journals. This brings up

___      _         __ _ _           ___  ___    ___

/   \__ _| |_ __ _ / _(_) | ___     /   \/ __\  / _ \

/ /\ / _` | __/ _` | |_| | |/ _ \   / /\ /__\// / /_\/

/ /_// (_| | || (_| |  _| | |  __/  / /_// \/  \/ /_\\

/___,' \__,_|\__\__,_|_| |_|_|\___| /___,'\_____/\____/

Available collections:

0: _structures

1: _users

2: _routing

3: _modules

4: _graphs

5: products

6: prices

*: all

Collection to check:

You can now select which database and collection you want to check. After you selected one or all of the collections, a consistency check will be performed.

Checking collection #1: _users

Database

path: /usr/local/var/lib/arangodb

Collection

name: _users

identifier: 82343

Datafiles

# of journals: 1

# of compactors: 1

# of datafiles: 0

Datafile

path: /usr/local/var/lib/arangodb/collection-82343/journal-1065383.db

type: journal

current size: 33554432

maximal size: 33554432

total used: 256

# of entries: 3

status: OK

If there is a problem with one of the datafiles, then the database debugger will print it and prompt for whether to attempt to fix it.

WARNING: The journal was not closed properly, the last entries are corrupted.

This might happen ArangoDB was killed and the last entries were not

fully written to disk.

Wipe the last entries (Y/N)?

If you answer Y, the corrupted entry will be removed.

If you see a corruption in a datafile (and not a journal), then something is terribly wrong. These files are immutable and never changed by ArangoDB. A corruption in such file is an indication of a hard-disk failure.

Naming Conventions in ArangoDB

The following naming conventions should be followed by users when creating databases, collections and documents in ArangoDB.

Database Names

ArangoDB will always start up with a default database, named _system. Users can create additional databases in ArangoDB, provided the database names conform to the following constraints:

- Database names must only consist of the letters a to z (both lower and upper case allowed), the numbers 0 to 9, and the underscore (_) or dash (-) symbols This also means that any non-ASCII database names are not allowed

- Database names must always start with a letter. Database names starting with an underscore are considered to be system databases, and users should not create or delete those

- The maximum allowed length of a database name is 64 bytes

- Database names are case-sensitive

Collection Names

Users can pick names for their collections as desired, provided the following naming constraints are not violated:

- Collection names must only consist of the letters a to z (both in lower and upper case), the numbers 0 to 9, and the underscore (_) or dash (-) symbols. This also means that any non-ASCII collection names are not allowed

- User-defined collection names must always start with a letter. System collection names must start with an underscore. All collection names starting with an underscore are considered to be system collections that are for ArangoDB's internal use only. System collection names should not be used by end users for their own collections

- The maximum allowed length of a collection name is 64 bytes

- Collection names are case-sensitive

Document Keys

Users can define their own keys for documents they save. The document key will be saved along with a document in the _key attribute. Users can pick key values as required, provided that the values conform to the following restrictions:

- The key must be a string value. Numeric keys are not allowed, but any numeric value can be put into a string and can then be used as document key.

- The key must be at least 1 byte and at most 254 bytes long. Empty keys are disallowed when specified (though it may be valid to completely omit the _key attribute from a document)

- It must consist of the letters a-z (lower or upper case), the digits 0-9 or any of the following punctuation characters:  _   -   :   .   @   (   )   +   ,   =   ;   $   !   *   '   %

- Any other characters, especially multi-byte UTF-8 sequences, whitespace or punctuation characters cannot be used inside key values

- The key must be unique within the collection it is used

Keys are case-sensitive, i.e. myKey and MyKEY are considered to be different keys.

Specifying a document key is optional when creating new documents. If no document key is specified by the user, ArangoDB will create the document key itself as each document is required to have a key.

There are no guarantees about the format and pattern of auto-generated document keys other than the above restrictions. Clients should therefore treat auto-generated document keys as opaque values and not rely on their format.

Attribute Names

Users can pick attribute names for document attributes as desired, provided the following attribute naming constraints are not violated:

- Attribute names starting with an underscore are considered to be system attributes for ArangoDB's internal use. Such attribute names are already used by ArangoDB for special purposes:

- _id is used to contain a document's handle

- _key is used to contain a document's user-defined key

- _rev is used to contain the document's revision number

- In edge collections, the

- _from

- _to

attributes are used to reference other documents.

More system attributes may be added in the future without further notice so end users should try to avoid using their own attribute names starting with underscores.

- Theoretically, attribute names can include punctuation and special characters as desired, provided the name is a valid UTF-8 string. For maximum portability, special characters should be avoided though. For example, attribute names may contain the dot symbol, but the dot has a special meaning in JavaScript and also in AQL, so when using such attribute names in one of these languages, the attribute name would need to be quoted by the end user. This will work but requires more work so it might be better to use attribute names which don't require any quoting/escaping in all languages used. This includes languages used by the client (e.g. Ruby, PHP) if the attributes are mapped to object members there.

- Attribute names starting with an at-mark (@) will need to be enclosed in backticks when used in an AQL query to tell them apart from bind variables. Therefore we do not encourage the use of attributes starting with at-marks, though they will work when used properly.

- ArangoDB does not enforce a length limit for attribute names. However, long attribute names may use more memory in result sets etc. Therefore the use of long attribute names is discouraged.

- As ArangoDB saves document attribute names separate from the actual document attribute value data, the combined length of all attribute names for a document must fit into an ArangoDB shape structure. The maximum combined names length is variable and depends on the number and data types of attributes used.

- Attribute names are case-sensitive.

- Attribute names should not include the NUL byte ( \u0000 ), as names will be truncated at the first NUL byte.

- Attributes with empty names (the empty string) are removed from the document when saving it.

When the document is later requested, it will be returned without these attributes. For example, if this document is saved:

{ "a" : 1, "" : 2, "b": 3 }

and later requested, it will be returned like this:

{ "a" : 1, "b": 3 }

Error codes and meanings

general error messages

0 - no error: No error has occurred.

1 - failed: Will be raised when a general error occurred.

2 - system error: Will be raised when operating system error occurred.

3 - out of memory: Will be raised when there is a memory shortage.

4 - internal error: Will be raised when an internal error occurred.

5 - illegal number: Will be raised when an illegal representation of a number was given.

6 - numeric overflow: Will be raised when a numeric overflow occurred.

7 - illegal option: Will be raised when an unknown option was supplied by the user.

8 - dead process identifier: Will be raised when a PID without a living process was found.

9 - not implemented: Will be raised when hitting an unimplemented feature.

10 - bad parameter: Will be raised when the parameter does not fulfill the requirements.

11 - forbidden: Will be raised when you are missing permission for the operation.

12 - out of memory in mmap: Will be raised when there is a memory shortage.

13 - csv is corrupt: Will be raised when encountering a corrupt csv line.

14 - file not found: Will be raised when a file is not found.

15 - cannot write file: Will be raised when a file cannot be written.

16 - cannot overwrite file: Will be raised when an attempt is made to overwrite an existing file.

17 - type error: Will be raised when a type error is unencountered.

18 - lock timeout: Will be raised when there's a timeout waiting for a lock.

19 - cannot create directory: Will be raised when an attempt to create a directory fails.

20 - cannot create temporary file: Will be raised when an attempt to create a temporary file fails.

21 - canceled request: Will be raised when a request is canceled by the user.

22 - intentional debug error: Will be raised intentionally during debugging.

23 - internal error with attribute ID in shaper: Will be raised if an attribute ID is not found in the shaper but should have been.

24 - internal error if a legend could not be created: Will be raised if the legend generator was only given access to the shape and some sids are in the data object (inhomogeneous lists).

25 - IP address is invalid: Will be raised when the structure of an IP address is invalid.

26 - internal error if a legend for a marker does not yet exist in the same WAL file: Will be raised internally, then fixed internally, and never come out to the user.

27 - file exists: Will be raised when a file already exists.

28 - locked: Will be raised when a resource or an operation is locked.

29 - deadlock detected: Will be raised when a deadlock is detected when accessing collections.

HTTP standard errors

400 - bad parameter: Will be raised when the HTTP request does not fulfill the requirements.

401 - unauthorized: Will be raised when authorization is required but the user is not authorized.

403 - forbidden: Will be raised when the operation is forbidden.

404 - not found: Will be raised when an URI is unknown.

405 - method not supported: Will be raised when an unsupported HTTP method is used for an operation.

412 - precondition failed: Will be raised when a precondition for an HTTP request is not met.

500 - internal server error: Will be raised when an internal server is encountered.

HTTP errors

600 - invalid JSON object: Will be raised when a string representation of a JSON object is corrupt.

601 - superfluous URL suffices: Will be raised when the URL contains superfluous suffices.

ArangoDB internal storage errors

For errors that occur because of a programming error.

1000 - illegal state: Internal error that will be raised when the datafile is not in the required state.

1001 - could not shape document: Internal error that will be raised when the shaper encountered a problem.

1002 - datafile sealed: Internal error that will be raised when trying to write to a datafile.

1003 - unknown type: Internal error that will be raised when an unknown collection type is encountered.

1004 - read only: Internal error that will be raised when trying to write to a read-only datafile or collection.

1005 - duplicate identifier: Internal error that will be raised when a identifier duplicate is detected.

1006 - datafile unreadable: Internal error that will be raised when a datafile is unreadable.

1007 - datafile empty: Internal error that will be raised when a datafile is empty.

1008 - logfile recovery error: Will be raised when an error occurred during WAL log file recovery.

ArangoDB storage errors

For errors that occur because of an outside event.

1100 - corrupted datafile: Will be raised when a corruption is detected in a datafile.

1101 - illegal or unreadable parameter file: Will be raised if a parameter file is corrupted or cannot be read.

1102 - corrupted collection: Will be raised when a collection contains one or more corrupted data files.

1103 - mmap failed: Will be raised when the system call mmap failed.

1104 - filesystem full: Will be raised when the filesystem is full.

1105 - no journal: Will be raised when a journal cannot be created.

1106 - cannot create/rename datafile because it already exists: Will be raised when the datafile cannot be created or renamed because a file of the same name already exists.

1107 - database directory is locked: Will be raised when the database directory is locked by a different process.

1108 - cannot create/rename collection because directory already exists: Will be raised when the collection cannot be created because a directory of the same name already exists.

1109 - msync failed: Will be raised when the system call msync failed.

1110 - cannot lock database directory: Will be raised when the server cannot lock the database directory on startup.

1111 - sync timeout: Will be raised when the server waited too long for a datafile to be synced to disk.

ArangoDB storage errors

For errors that occur when fulfilling a user request.

1200 - conflict: Will be raised when updating or deleting a document and a conflict has been detected.

1201 - invalid database directory: Will be raised when a non-existing database directory was specified when starting the database.

1202 - document not found: Will be raised when a document with a given identifier or handle is unknown.

1203 - collection not found: Will be raised when a collection with a given identifier or name is unknown.

1204 - parameter 'collection' not found: Will be raised when the collection parameter is missing.

1205 - illegal document handle: Will be raised when a document handle is corrupt.

1206 - maximal size of journal too small: Will be raised when the maximal size of the journal is too small.

1207 - duplicate name: Will be raised when a name duplicate is detected.

1208 - illegal name: Will be raised when an illegal name is detected.

1209 - no suitable index known: Will be raised when no suitable index for the query is known.

1210 - unique constraint violated: Will be raised when there is a unique constraint violation.

1212 - index not found: Will be raised when an index with a given identifier is unknown.

1213 - cross collection request not allowed: Will be raised when a cross-collection is requested.

1214 - illegal index handle: Will be raised when a index handle is corrupt.

1215 - cap constraint already defined: Will be raised when a cap constraint was already defined.

1216 - document too large: Will be raised when the document cannot fit into any datafile because of it is too large.

1217 - collection must be unloaded: Will be raised when a collection should be unloaded, but has a different status.

1218 - collection type invalid: Will be raised when an invalid collection type is used in a request.

1219 - validator failed: Will be raised when the validation of an attribute of a structure failed.

1220 - parsing definition failed: Will be raised when parsing an attribute name definition failed.

1221 - illegal document key: Will be raised when a document key is corrupt.

1222 - unexpected document key: Will be raised when a user-defined document key is supplied for collections with auto key generation.

1224 - server database directory not writable: Will be raised when the server's database directory is not writable for the current user.

1225 - out of keys: Will be raised when a key generator runs out of keys.

1226 - missing document key: Will be raised when a document key is missing.

1227 - invalid document type: Will be raised when there is an attempt to create a document with an invalid type.

1228 - database not found: Will be raised when a non-existing database is accessed.

1229 - database name invalid: Will be raised when an invalid database name is used.

1230 - operation only allowed in system database: Will be raised when an operation is requested in a database other than the system database.

1231 - endpoint not found: Will be raised when there is an attempt to delete a non-existing endpoint.

1232 - invalid key generator: Will be raised when an invalid key generator description is used.

1233 - edge attribute missing: will be raised when the _from or _to values of an edge are undefined or contain an invalid value.

1234 - index insertion warning - attribute missing in document: Will be raised when an attempt to insert a document into an index is caused by in the document not having one or more attributes which the index is built on.

1235 - index creation failed: Will be raised when an attempt to create an index has failed.

1236 - write-throttling timeout: Will be raised when the server is write-throttled and a write operation has waited too long for the server to process queued operations.

1237 - collection type mismatch: Will be raised when a collection has a different type from what has been expected.

1238 - collection not loaded: Will be raised when a collection is accessed that is not yet loaded.

ArangoDB storage errors

For errors that occur but are anticipated.

1300 - datafile full: Will be raised when the datafile reaches its limit.

1301 - server database directory is empty: Will be raised when encountering an empty server database directory.

ArangoDB replication errors

1400 - no response: Will be raised when the replication applier does not receive any or an incomplete response from the master.

1401 - invalid response: Will be raised when the replication applier receives an invalid response from the master.

1402 - master error: Will be raised when the replication applier receives a server error from the master.

1403 - master incompatible: Will be raised when the replication applier connects to a master that has an incompatible version.

1404 - master change: Will be raised when the replication applier connects to a different master than before.

1405 - loop detected: Will be raised when the replication applier is asked to connect to itself for replication.

1406 - unexpected marker: Will be raised when an unexpected marker is found in the replication log stream.

1407 - invalid applier state: Will be raised when an invalid replication applier state file is found.

1408 - invalid transaction: Will be raised when an unexpected transaction id is found.

1410 - invalid replication applier configuration: Will be raised when the configuration for the replication applier is invalid.

1411 - cannot perform operation while applier is running: Will be raised when there is an attempt to perform an operation while the replication applier is running.

1412 - replication stopped: Special error code used to indicate the replication applier was stopped by a user.

1413 - no start tick: Will be raised when the replication applier is started without a known start tick value.

1414 - start tick not present: Will be raised when the replication applier fetches data using a start tick, but that start tick is not present on the logger server anymore.

ArangoDB cluster errors

1450 - could not connect to agency: Will be raised when none of the agency servers can be connected to.

1451 - missing coordinator header: Will be raised when a DB server in a cluster receives a HTTP request without a coordinator header.

1452 - could not lock plan in agency: Will be raised when a coordinator in a cluster cannot lock the Plan hierarchy in the agency.

1453 - collection ID already exists: Will be raised when a coordinator in a cluster tries to create a collection and the collection ID already exists.

1454 - could not create collection in plan: Will be raised when a coordinator in a cluster cannot create an entry for a new collection in the Plan hierarchy in the agency.

1455 - could not read version in current in agency: Will be raised when a coordinator in a cluster cannot read the Version entry in the Current hierarchy in the agency.

1456 - could not create collection: Will be raised when a coordinator in a cluster notices that some DBServers report problems when creating shards for a new collection.

1457 - timeout in cluster operation: Will be raised when a coordinator in a cluster runs into a timeout for some cluster wide operation.

1458 - could not remove collection from plan: Will be raised when a coordinator in a cluster cannot remove an entry for a collection in the Plan hierarchy in the agency.

1459 - could not remove collection from current: Will be raised when a coordinator in a cluster cannot remove an entry for a collection in the Current hierarchy in the agency.

1460 - could not create database in plan: Will be raised when a coordinator in a cluster cannot create an entry for a new database in the Plan hierarchy in the agency.

1461 - could not create database: Will be raised when a coordinator in a cluster notices that some DBServers report problems when creating databases for a new cluster wide database.

1462 - could not remove database from plan: Will be raised when a coordinator in a cluster cannot remove an entry for a database in the Plan hierarchy in the agency.

1463 - could not remove database from current: Will be raised when a coordinator in a cluster cannot remove an entry for a database in the Current hierarchy in the agency.

1464 - no responsible shard found: Will be raised when a coordinator in a cluster cannot determine the shard that is responsible for a given document.

1465 - cluster internal HTTP connection broken: Will be raised when a coordinator in a cluster loses an HTTP connection to a DBserver in the cluster whilst transferring data.

1466 - must not specify _key for this collection: Will be raised when a coordinator in a cluster finds that the _key attribute was specified in a sharded collection the uses not only _key as sharding attribute.

1467 - got contradicting answers from different shards: Will be raised if a coordinator in a cluster gets conflicting results from different shards, which should never happen.

1468 - not all sharding attributes given: Will be raised if a coordinator tries to find out which shard is responsible for a partial document, but cannot do this because not all sharding attributes are specified.

1469 - must not change the value of a shard key attribute: Will be raised if there is an attempt to update the value of a shard attribute.

1470 - unsupported operation or parameter: Will be raised when there is an attempt to carry out an operation that is not supported in the context of a sharded collection.

1471 - this operation is only valid on a coordinator in a cluster: Will be raised if there is an attempt to run a coordinator-only operation on a different type of node.

1472 - error reading Plan in agency: Will be raised if a coordinator or DBserver cannot read the Plan in the agency.

1473 - could not truncate collection: Will be raised if a coordinator cannot truncate all shards of a cluster collection.

1474 - error in cluster internal communication for AQL: Will be raised if the internal communication of the cluster for AQL produces an error.

1475 - document not found or sharding attributes changed: Will be raised when a document with a given identifier or handle is unknown, or if the sharding attributes have been changed in a REPLACE operation in the cluster.

1476 - could not determine my ID from my local info: Will be raised if a cluster server at startup could not determine its own ID from the local info provided.

ArangoDB query errors

1500 - query killed: Will be raised when a running query is killed by an explicit admin command.

1501 - %s: Will be raised when query is parsed and is found to be syntactically invalid.

1502 - query is empty: Will be raised when an empty query is specified.

1503 - runtime error '%s': Will be raised when a runtime error is caused by the query.

1504 - number out of range: Will be raised when a number is outside the expected range.

1510 - variable name '%s' has an invalid format: Will be raised when an invalid variable name is used.

1511 - variable '%s' is assigned multiple times: Will be raised when a variable gets re-assigned in a query.

1512 - unknown variable '%s': Will be raised when an unknown variable is used or the variable is undefined the context it is used.

1521 - unable to read-lock collection %s: Will be raised when a read lock on the collection cannot be acquired.

1522 - too many collections: Will be raised when the number of collections in a query is beyond the allowed value.

1530 - document attribute '%s' is assigned multiple times: Will be raised when a document attribute is re-assigned.

1540 - usage of unknown function '%s()': Will be raised when an undefined function is called.

1541 - invalid number of arguments for function '%s()', expected number of arguments: minimum: %d, maximum: %d: Will be raised when the number of arguments used in a function call does not match the expected number of arguments for the function.

1542 - invalid argument type in call to function '%s()': Will be raised when the type of an argument used in a function call does not match the expected argument type.

1543 - invalid regex value: Will be raised when an invalid regex argument value is used in a call to a function that expects a regex.

1550 - invalid structure of bind parameters: Will be raised when the structure of bind parameters passed has an unexpected format.

1551 - no value specified for declared bind parameter '%s': Will be raised when a bind parameter was declared in the query but the query is being executed with no value for that parameter.

1552 - bind parameter '%s' was not declared in the query: Will be raised when a value gets specified for an undeclared bind parameter.

1553 - bind parameter '%s' has an invalid value or type: Will be raised when a bind parameter has an invalid value or type.

1560 - invalid logical value: Will be raised when a non-boolean value is used in a logical operation.

1561 - invalid arithmetic value: Will be raised when a non-numeric value is used in an arithmetic operation.

1562 - division by zero: Will be raised when there is an attempt to divide by zero.

1563 - array expected: Will be raised when a non-array operand is used for an operation that expects an array argument operand.

1569 - FAIL(%s) called: Will be raised when the function FAIL() is called from inside a query.

1570 - no suitable geo index found for geo restriction on '%s': Will be raised when a geo restriction was specified but no suitable geo index is found to resolve it.

1571 - no suitable fulltext index found for fulltext query on '%s': Will be raised when a fulltext query is performed on a collection without a suitable fulltext index.

1572 - invalid date value: Will be raised when a value cannot be converted to a date.

ERROR_QUERY_MULTI_MODIFY,1573,"multi-modify query", "Will be raised when an AQL query contains more than one data-modifying operation."

ERROR_QUERY_INVALID_AGGREGATE_EXPRESSION,1574,"invalid aggregate expression", "Will be raised when an AQL query contains an invalid aggregate expression."

ERROR_QUERY_COMPILE_TIME_OPTIONS,1575,"query options must be readable at query compile time", "Will be raised when an AQL data-modification query contains options that cannot be figured out at query compile time."

ERROR_QUERY_EXCEPTION_OPTIONS,1576,"query options expected", "Will be raised when an AQL data-modification query contains an invalid options specification."

ERROR_QUERY_COLLECTION_USED_IN_EXPRESSION,1577,"collection '%s' used as expression operand", "Will be raised when a collection is used as an operand in an AQL expression."

ERROR_QUERY_DISALLOWED_DYNAMIC_CALL,1578,"disallowed dynamic call to '%s'", "Will be raised when a dynamic function call is made to a function that cannot be called dynamically."

ERROR_QUERY_ACCESS_AFTER_MODIFICATION,1579,"access after data-modification", "Will be raised when collection data are accessed after a data-modification operation."

AQL user functions

1580 - invalid user function name: Will be raised when a user function with an invalid name is registered.

1581 - invalid user function code: Will be raised when a user function is registered with invalid code.

1582 - user function '%s()' not found: Will be raised when a user function is accessed but not found.

1583 - user function runtime error: %s: Will be raised when a user function throws a runtime exception.

AQL query registry

ERROR_QUERY_BAD_JSON_PLAN,1590,"bad execution plan JSON", "Will be raised when an HTTP API for a query got an invalid JSON object."

ERROR_QUERY_NOT_FOUND,1591,"query ID not found", "Will be raised when an Id of a query is not found by the HTTP API."

ERROR_QUERY_IN_USE,1592,"query with this ID is in use", "Will be raised when an Id of a query is found by the HTTP API but the query is in use."

ArangoDB cursor errors

1600 - cursor not found: Will be raised when a cursor is requested via its id but a cursor with that id cannot be found.

1601 - cursor is busy: Will be raised when a cursor is requested via its id but a concurrent request is still using the cursor.

ArangoDB transaction errors

1650 - internal transaction error: Will be raised when a wrong usage of transactions is detected. this is an internal error and indicates a bug in ArangoDB.

1651 - nested transactions detected: Will be raised when transactions are nested.

1652 - unregistered collection used in transaction: Will be raised when a collection is used in the middle of a transaction but was not registered at transaction start.

1653 - disallowed operation inside transaction: Will be raised when a disallowed operation is carried out in a transaction.

1654 - transaction aborted: Will be raised when a transaction was aborted.

User management

1700 - invalid user name: Will be raised when an invalid user name is used.

1701 - invalid password: Will be raised when an invalid password is used.

1702 - duplicate user: Will be raised when a user name already exists.

1703 - user not found: Will be raised when a user name is updated that does not exist.

1704 - user must change his password: Will be raised when the user must change his password.

Application management

1750 - invalid application name: Will be raised when an invalid application name is specified.

1751 - invalid mount: Will be raised when an invalid mount is specified.

1752 - application download failed: Will be raised when an application download from the central repository failed.

1753 - application upload failed: Will be raised when an application upload from the client to the ArangoDB server failed.

Key value access

1800 - invalid key declaration: Will be raised when an invalid key specification is passed to the server

1801 - key already exists: Will be raised when a key is to be created that already exists

1802 - key not found: Will be raised when the specified key is not found

1803 - key is not unique: Will be raised when the specified key is not unique

1804 - key value not changed: Will be raised when updating the value for a key does not work

1805 - key value not removed: Will be raised when deleting a key/value pair does not work

1806 - missing value: Will be raised when the value is missing

Task errors

1850 - invalid task id: Will be raised when a task is created with an invalid id.

1851 - duplicate task id: Will be raised when a task id is created with a duplicate id.

1852 - task not found: Will be raised when a task with the specified id could not be found.

Graph / traversal errors

1901 - invalid graph: Will be raised when an invalid name is passed to the server.

1902 - could not create graph: Will be raised when an invalid name, vertices or edges is passed to the server.

1903 - invalid vertex: Will be raised when an invalid vertex id is passed to the server.

1904 - could not create vertex: Will be raised when the vertex could not be created.

1905 - could not change vertex: Will be raised when the vertex could not be changed.

1906 - invalid edge: Will be raised when an invalid edge id is passed to the server.

1907 - could not create edge: Will be raised when the edge could not be created.

1908 - could not change edge: Will be raised when the edge could not be changed.

1909 - too many iterations - try increasing the value of 'maxIterations': Will be raised when too many iterations are done in a graph traversal.

1910 - invalid filter result: Will be raised when an invalid filter result is returned in a graph traversal.

1920 - multi use of edge collection in edge def: an edge collection may only be used once in one edge definition of a graph.,

1921 - edge collection already used in edge def: is already used by another graph in a different edge definition.,

1922 - missing graph name: a graph name is required to create a graph.,

1923 - malformed edge definition: the edge definition is malformed. It has to be an array of objects.,

1924 - graph not found: a graph with this name could not be found.,

1925 - graph already exists: a graph with this name already exists.,

1926 - vertex collection does not exist or is not part of the graph: the specified vertex collection does not exist or is not part of the graph.,

1927 - not a vertex collection: the collection is not a vertex collection.,

1928 - not in orphan collection: Vertex collection not in orphan collection of the graph.,

1929 - collection already used in edge def: The collection is already used in an edge definition of the graph.,

1930 - edge collection not used in graph: The edge collection is not used in any edge definition of the graph.,

1931 - is not an ArangoCollection: The collection is not an ArangoCollection.,

1932 - collection _graphs does not exist: collection _graphs does not exist.,

1933 - Invalid example type. Has to be String, Array or Object: Invalid example type. Has to be String, Array or Object.,

1934 - Invalid example type. Has to be Array or Object: Invalid example type. Has to be Array or Object.,

1935 - Invalid number of arguments. Expected: : Invalid number of arguments. Expected: ,

1936 - Invalid parameter type.: Invalid parameter type.,

1937 - Invalid id: Invalid id,

1938 - collection used in orphans: The collection is already used in the orphans of the graph.,

1939 - edge collection does not exist or is not part of the graph: the specified edge collection does not exist or is not part of the graph.,

1940 - empty graph: The requested graph has no edge collections.

Session errors

1950 - unknown session: Will be raised when an invalid/unknown session id is passed to the server.

1951 - session expired: Will be raised when a session is expired.

Simple Client

2000 - unknown client error: This error should not happen.

2001 - could not connect to server: Will be raised when the client could not connect to the server.

2002 - could not write to server: Will be raised when the client could not write data.

2003 - could not read from server: Will be raised when the client could not read data.

Foxx Manager

3000 - malformed manifest file: The manifest file is malformed. It is not in a valid JSON format.

3001 - manifest file is invalid: The manifest file of this application is invalid.

3002 - missing manifest attribute: The manifest file is incomplete. A required attribute is missing.

3003 - unable to extract app root path: The root path of the application could not be found.

3004 - invalid foxx options: The options used to configure the foxx are invalid.

3005 - failed to execute script: The script provided contains errors.

3006 - syntax error in script: contains a syntax error:

3007 - mountpoint is invalid: mountpoint is invalid

3008 - No foxx found at this location: No foxx found at this location

3009 - App not found: No app found at this mountpoint

3010 - App not configured: The app has to be configured before it can be used

JavaScript module loader

3100 - cannot locate module: The module path could not be resolved.

3101 - syntax error in module: The module could not be parsed because of a syntax error.

3102 - failed to wrap module: The module wrapper could not be generated. This may indicate a problem with some of the names of the module's context variables.

3103 - failed to invoke module: Failed to invoke the module in its context.

3110 - unknown file type: The module path resolves to a file of an unknown type.

3111 - path must be absolute: The module path must be absolute.

3112 - cannot use '..' to escape top-level-directory: The relative module path can not escape the module's top-level directory.

3113 - drive local path is not supported: The module path contains a Windows drive letter, which is not supported.

3120 - corrupted module origin: The module origin is invalid.

3121 - corrupted package origin: The package origin is invalid.

3125 - no content: The module resolves to a document which is empty or malformed.

3130 - cannot read main file: The module's main file is not readable.

3131 - main file is not of type 'js': The module's main file is not a JavaScript file.

results, which are not errors

10000 - element not inserted into structure, because it already exists: Will be returned if the element was not insert because it already exists.

10001 - element not found in structure: Will be returned if the element was not found in the structure.

foxx app update via github

20000 - newest version of app already installed: newest version of app already installed

dispatcher errors

ERROR_QUEUE_ALREADY_EXISTS,21000,"named queue already exists", "Will be returned if a queue with this name already exists."

21001 - dispatcher stopped: Will be returned if a shutdown is in progress.

21002 - named queue does not exist: Will be returned if a queue with this name does not exist.

21003 - named queue is full: Will be returned if a queue with this name is full.

Glossary

Collection

A collection consists of documents. It is uniquely identified by its collection identifier. It also has a unique name that clients should use to identify and access it. Collections can be renamed. It will change the collection name, but not the collection identifier. Collections contain documents of a specific type. There are currently two types: document (default) and edge. The type is specified by the user when the collection is created, and cannot be changed later.

Collection Identifier

A collection identifier identifies a collection in a database. It is a string value and is unique within the database. Up to including ArangoDB 1.1, the collection identifier has been a client's primary means to access collections. Starting with ArangoDB 1.2, clients should instead use a collection's unique name to access a collection instead of its identifier.

ArangoDB currently uses 64bit unsigned integer values to maintain collection ids internally. When returning collection ids to clients, ArangoDB will put them into a string to ensure the collection id is not clipped by clients that do not support big integers. Clients should treat the collection ids returned by ArangoDB as opaque strings when they store or use it locally.

Collection Name

A collection name identifies a collection in a database. It is a string and is unique within the database. Unlike the collection identifier it is supplied by the creator of the collection. The collection name must consist of letters, digits, and the _ (underscore) and - (dash) characters only. Please refer to NamingConventions for more information on valid collection names.

Database

ArangoDB can handle multiple databases in the same server instance. Databases can be used to logically group and separate data. An ArangoDB database consists of collections and dedicated database-specific worker processes.

A database contains its own collections (which cannot be accessed from other databases), Foxx applications, and replication loggers and appliers. Each ArangoDB database contains its own system collections (e.g. _users, _replication, ...).

There will always be at least one database in ArangoDB. This is the default database, named _system. This database cannot be dropped, and provides special operations for creating, dropping, and enumerating databases. Users can create additional databases and give them unique names to access them later. Database management operations cannot be initiated from out of user-defined databases.

When ArangoDB is accessed via its HTTP REST API, the database name is read from the first part of the request URI path (e.g. /_db/_system/...). If the request URI does not contain a database name, the database name is automatically derived from the endpoint. Please refer to DatabaseEndpoint for more information.

Database Name

A single ArangoDB instance can handle multiple databases in parallel. When multiple databases are used, each database must be given a unique name. This name is used to uniquely identify a database. The default database in ArangoDB is named _system.

The database name is a string consisting of only letters, digits and the _ (underscore) and - (dash) characters. User-defined database names must always start with a letter. Database names is case-sensitive.

Database Organization

A single ArangoDB instance can handle multiple databases in parallel. By default, there will be at least one database, which is named _system.

Databases are physically stored in separate sub-directories underneath the database directory, which itself resides in the instance's data directory.

Each database has its own sub-directory, named database-. The database directory contains sub-directories for the collections of the database, and a file named parameter.json. This file contains the database id and name.

In an example ArangoDB instance which has two databases, the filesystem layout could look like this:

data/                     # the instance's data directory

databases/              # sub-directory containing all databases' data

database-<id>/        # sub-directory for a single database

parameter.json      # file containing database id and name

collection-<id>/    # directory containing data about a collection

database-<id>/        # sub-directory for another database

parameter.json      # file containing database id and name

collection-<id>/    # directory containing data about a collection

collection-<id>/    # directory containing data about a collection

Foxx applications are also organized in database-specific directories inside the application path. The filesystem layout could look like this:

apps/                   # the instance's application directory

system/               # system applications (can be ignored)

_db/                  # sub-directory containing database-specific applications

<database-name>/    # sub-directory for a single database

<mountpoint>/APP  # sub-directory for a single application

<mountpoint>/APP  # sub-directory for a single application

<database-name>/    # sub-directory for another database

<mountpoint>/APP  # sub-directory for a single application

Document

Documents in ArangoDB are JSON objects. These objects can be nested (to any depth) and may contain arrays. Each document is uniquely identified by its document handle.

Document ETag

The document revision ( _rev  value) enclosed in double quotes. The revision is returned by several HTTP API methods in the Etag HTTP header.

Document Handle

A document handle uniquely identifies a document in the database. It is a string and consists of the collection's name and the document key ( _key  attribute) separated by /. The document handle is stored in a document's  _id  attribute.

Document Key

A document key is a string that uniquely identifies a document in a given collection. It can and should be used by clients when specific documents are searched. Document keys are stored in the  _key  attribute of documents. The key values are automatically indexed by ArangoDB in a collection's primary index. Thus looking up a document by its key is regularly a fast operation. The  _key  value of a document is immutable once the document has been created.

By default, ArangoDB will auto-generate a document key if no  _key  attribute is specified, and use the user-specified  _key  value otherwise.

This behavior can be changed on a per-collection level by creating collections with the  keyOptions  attribute.

Using  keyOptions  it is possible to disallow user-specified keys completely, or to force a specific regime for auto-generating the  _key  values.

There are some restrictions for user-defined keys (see NamingConventions for document keys.

Document Revision

As ArangoDB supports MVCC, documents can exist in more than one revision. The document revision is the MVCC token used to identify a particular revision of a document. It is a string value currently containing an integer number and is unique within the list of document revisions for a single document. Document revisions can be used to conditionally update, replace or delete documents in the database. In order to find a particular revision of a document, you need the document handle and the document revision.

The document revision is stored in the  _rev  attribute of a document, and is set and updated by ArangoDB automatically. The  _rev  value cannot be set from the outside.

ArangoDB currently uses 64bit unsigned integer values to maintain document revisions internally. When returning document revisions to clients, ArangoDB will put them into a string to ensure the revision id is not clipped by clients that do not support big integers. Clients should treat the revision id returned by ArangoDB as an opaque string when they store or use it locally. This will allow ArangoDB to change the format of revision ids later if this should be required. Clients can use revisions ids to perform simple equality/non-equality comparisons (e.g. to check whether a document has changed or not), but they should not use revision ids to perform greater/less than comparisons with them to check if a document revision is older than one another, even if this might work for some cases.

Edge

Edges are special documents used for connecting other documents into a graph. An edge describes the connection between two documents using the internal attributes: _from and _to. These contain document handles, namely the start-point and the end-point of the edge.

The values of _from and _to are immutable once saved.

Edge Collection

Edge collections are collections that store edges.

Edge Definition

Edge definitions are parts of the definition of  named grahps . They describe which edge collections connect which vertex collections.

General Graph

Module maintaining graph setup in the  _graphs  collection - aka  named graphs . Configures which edge collections relate to which vertex collections. Ensures graph consistency in modification queries.

Named Graphs

Named graphs enforce consistency between edge collections and vertex collections, so if you remove a vertex, edges pointing to it will be removed too.

Index

Indexes are used to allow fast access to documents in a collection. All collections have a primary index, which is the document's _key attribute. This index cannot be dropped or changed.

Edge collections will also have an automatically created edges index, which cannot be modified. This index provides quick access to documents via the _from and _to attributes.

Most user-land indexes can be created by defining the names of the attributes which should be indexed. Some index types allow indexing just one attribute (e.g. fulltext index) whereas other index types allow indexing multiple attributes.

Indexing system attributes such as _id, _key, _from, and _to in user-defined indexes is not supported by any index type. Manually creating an index that relies on any of these attributes is unsupported.

Edges Index

An edges index is automatically created for edge collections. It contains connections between vertex documents and is invoked when the connecting edges of a vertex are queried. There is no way to explicitly create or delete edge indexes.

Fulltext Index

A fulltext index can be used to find words, or prefixes of words inside documents. A fulltext index can be defined on one attribute only, and will include all words contained in documents that have a textual value in the index attribute. Since ArangoDB 2.6 the index will also include words from the index attribute if the index attribute is an array of strings, or an object with string value members.

For example, given a fulltext index on the  translations  attribute and the following documents, then searching for  лиса  using the fulltext index would return only the first document. Searching for the index for the exact string  Fox  would return the first two documents, and searching for  prefix:Fox  would return all three documents:

{ translations: { en: "fox", de: "Fuchs", fr: "renard", ru: "лиса" } }

{ translations: "Fox is the English translation of the German word Fuchs" }

{ translations: [ "ArangoDB", "document", "database", "Foxx" ] }

If the index attribute is neither a string, an object or an array, its contents will not be indexed. When indexing the contents of an array attribute, an array member will only be included in the index if it is a string. When indexing the contents of an object attribute, an object member value will only be included in the index if it is a string. Other data types are ignored and not indexed.

Only words with a (specifiable) minimum length are indexed. Word tokenization is done using the word boundary analysis provided by libicu, which is taking into account the selected language provided at server start. Words are indexed in their lower-cased form. The index supports complete match queries (full words) and prefix queries.

Geo Index

A geo index is used to find places on the surface of the earth fast.

Index Handle

An index handle uniquely identifies an index in the database. It is a string and consists of a collection name and an index identifier separated by /.

Hash Index

A hash index is used to find documents based on examples. A hash index can be created for one or multiple document attributes.

A hash index will only be used by queries if all indexed attributes are present in the example or search query, and if all attributes are compared using the equality (== operator). That means the hash index does not support range queries.

A unique hash index has an amortized complexity of O(1) for lookup, insert, update, and remove operations. The non-unique hash index is similar, but amortized lookup performance is O(n), with n being the number of index entries with the same lookup value.

Skiplist Index

A skiplist is a sorted index type that can be used to find ranges of documents.

Anonymous Graphs

You may use edge collections with vertex collections without the graph management facilities. However, graph consistency is not enforced by these. If you remove vertices, you have to ensure by yourselves edges pointing to this vertex are removed. Anonymous graphs may not be browsed using graph viewer in the webinterface. This may be faster in some scenarios.

Table of Contents

Introduction

Installing

Linux

Mac OS X

Windows

ARM

Compiling

Upgrading

Incompatible changes in 2.8

Upgrading to 2.8

Incompatible changes in 2.7

Incompatible changes in 2.6

Upgrading to 2.6

Incompatible changes in 2.5

Upgrading to 2.5

Incompatible changes in 2.4

Upgrading to 2.4

Incompatible changes in 2.3

Upgrading to 2.3

Upgrading to 2.2

Cluster setup

Whats New

Whats New in 2.7

Whats New in 2.6

Whats New in 2.5

Whats New in 2.4

Whats New in 2.3

Whats New in 2.2

Whats New in 2.1

First Steps

Getting Familiar

The ArangoDB Server

The ArangoDB Shell

Shell Output

Configuration

Details

Collections

ArangoDB Web Interface

Creating and Dropping Databases

Working with Databases

Notes about Databases

Creating and Dropping Collections

Collection Methods

Database Methods

Working with Documents

Address and ETag

Collection Methods

Database Methods

Indexing

Index Basics

Which Index to use when

How ArangoDB uses indexes

Working with Indexes

Hash Indexes

Skiplists

Fulltext Indexes

Geo Indexes

Cap Constraint

Simple Queries

Sequential Access

Pagination

Modification Queries

Geo Queries

Fulltext Queries

Transactions

Transaction invocation

Passing parameters

Locking and isolation

Durability

Limitations

AQL

How to invoke AQL

Data modification queries

The AQL query optimizer

The AQL query result cache

Language Basics

Functions

Type cast

String

Numeric

Date

Array

Object / Document

Geo

Fulltext

Miscellaneous

Query Results

Operators

High level Operations

Graphs

Traversal

Named Operations

Other

Advanced Features

Extending AQL

Conventions

Registering Functions

AQL Examples

Collection based queries

Data-modification queries

Projections and filters

Joins

Grouping

Traversals

Graphs

General Graphs

Graph Management

Graph Functions

Fluent Query Interface

Traversals

Using Traversal Objects

Example Data

Working with Edges

Foxx

In a nutshell

Install

Github

Store

Local

Remote

Generate

Development

Dev-Mode

Debugging

Source Files

Controller

Scripts

Model

Repository

Queries

Sessions

Background Tasks

Console API

Meta information

Exports

Documentation

Production

Prod-Mode

Debugging

New Versions

Cluster

Install

Setup & Teardown

ArangoDB's Actions

Delivering HTML Pages

Json Objects

Modifying

Replication

Components

Example Setup

Syncing Collections

Replication Limitations

Replication Overhead

Sharding

How to try it out

Implementation

Authentication

Firewall setup

Server Configuration

Arangod options

Write-ahead log options

Managing Endpoints

Cluster options

Logging options

Communication options

Authentication

Emergency Console

HTTP API

General HTTP Handling

Databases

To-Endpoint

Management

Notes on Databases

Documents

Address and ETag

Working with

Edges

Address and ETag

Working with Edges

Exporting data

AQL Query Cursors

Query Results

Accessing Cursors

AQL Queries

AQL Query Cache

AQL User Functions Management

Simple Queries

Collections

Creating

Getting Information

Modifying

Indexes

Working with Indexes

Cap Constraints

Hash

Skiplist

Geo

Fulltext

Transactions

General Graph

Management

Vertices

Edges

Traversals

Replication

Replication Dump

Replication Logger

Replication Applier

Other Replication Commands

Tasks

Monitoring

User Management

Async Result

Endpoints

Sharding

Miscellaneous functions

Bulk Import / Export

JSON Documents HTTP-API

Headers & Values HTTP-API

Batch Requests HTTP-API

Arangoimp

Arangodump

Arangorestore

JavaScript Modules

"console"

"fs"

"process"

"request"

"actions"

"queries"

"planner"

Write-ahead log

Task Management

Using jsUnity

Administrating ArangoDB

Advanced

Arangob

Write-ahead log

Server Internals

Datafile Debugger

Naming Conventions

Database Names

Collection Names

Document Keys

Attribute Names

Error codes and meanings

Glossary