August 26, 2020

Introduction to the Roam Alpha API

RoamAlphaAPI is a javascript API available within Roam that allows querying of a Roam database using Datomic/Datalog.

This page is a summary of what I learned writing RoamPageSearch. I am not a clojure/datalog expert, and my javascript is pretty iffy, but all the examples work so hopefully this might be of use to some people.

Roam Alpha API exposes two datomic functions: q - allows querying of the Roam graph. pull- allows pulling of details of one block

Getting started with Roam Alpha API

The easiest way to get started is to try some examples in your browser developer console, after bringing up the Roam page in your browser. In Chrome, hit ⌘⌥+ I to bring up the developer tools , choose the console tab, and paste some of the examples below at the prompt >”.

You can navigate the results of the queries in the console by clicking on them to expand, or you can write javascript code to do interesting stuff with the results.

Each of the examples below can be copied and pasted into your Roam console. Note that the examples must be on one line.

A word of warning: if setting variables in the console, avoid short variable names as they may collide with variables Roam uses. If this does happen during your travels, just refresh the page.

You can paste the examples below into the console, however note that you need to use the version without line breaks.

Finding all pages

Find all blocks with a :node/title element (ie all pages), and return their database ids. window.roamAlphaAPI.q('[:find ?e :where [?e :node/title] ]')

In this example,
[?e :node/title] matches any block with a field called :node/title. ?e is a variable referring to the block that matches

The example returns an array of arrays- the outer array has an entry for each page, and the inner array has one element which is the database id of the page.

Getting Details of One Block

If you know the database id of one block, you can query all details of that block. For example, my Roam page is block 4 in my database. Running the following: window.roamAlphaAPI.pull('[*]',4) returns a javascript object similar to the following:

:node/title: "Roam"
:block/uid: "MOa3Jbhca"

:attrs/lookup: Array(3)
    0: {:db/id: 4}
    1: {:db/id: 2636}
    2: {:db/id: 11268}

:: Array(16)
    0: {:db/id: 2471}
    1: {:db/id: 3473}
    2: {:db/id: 3526}
    3: {:db/id: 3532}
    4: {:db/id: 3550}
    5: {:db/id: 11267}
    6: {:db/id: 11268}
    7: {:db/id: 23920}
    8: {:db/id: 23921}
    9: {:db/id: 23923}
    10: {:db/id: 26049}
    11: {:db/id: 26050}
    12: {:db/id: 26054}
    13: {:db/id: 30550}
    14: {:db/id: 30551}
    15: {:db/id: 30552}

:block/open: true

:children/view-type: ":bullet"
:create/email: ""
:create/time: 1580480550337
:db/id: 4
:edit/email: ""
:edit/time: 1597092215174
:entity/attrs: [Array(3)]

It’s worth highlighting the key fields and types of blocks. All blocks have the following :db/id- integer id representing id in database
:block/uid - 9 char string representing block uid, eg MOa3Jbhca” - Prepending this with will give a link to that page or block.

Blocks may have the following:

:block/children: which returns an array of blocks that are children of the target.

Pages have a special field: :node/title: which contains the page name

whereas all other blocks have a field called:

:block/string: which contains the block content as a string.

:block/refs: If a block references other blocks, then it may have this field which contains those blocks reference.

Putting q” and pull” together : the stupid way

So if you wanted a full representation of all pages in your database, you could run the following:

                      :find ?e  
                      :where [?e :node/title] ]

One line version: window.roamAlphaAPI.q('[ :find ?e :where [?e :node/title] ] ') .map(page=> window.roamAlphaAPI.pull('[*]',page[0]))

The q” part returns an array of pages reference, each of which is fed into a pull” expression to pull all fields. This returns an array of javascript object of pages, that we can do fun stuff with.

Putting q” and pull” together : Using Pull Parameters

However there is a better way of doing the above.

                          :find (pull ?e [*])  
                        :where [?e :node/title]]

One line version: window.roamAlphaAPI.q('[ :find (pull ?e [*]) :where [?e :node/title]] ')

This uses a pull expression (pull ?e [*])” which is invoked for each node, and pulls all fields defined as component fields”, which are whatever Roam has defined as fields to be pulled on each query.

Going Deeper: Pulling Children

If we drill into the output of the command above, we find something annoying. When a block points to other blocks (eg :block/children or :block/refs), we only have access to the dbid of those other blocks. For example, here is the object returned from the q/pull above showing one of my daily pages:

title: "July 27th, 2020"
uid: "07-27-2020"
children: Array(2)
0: {id: 27983}
1: {id: 28054}

What if we want to get the content of those children? We can always go back and do pulls for each block, but there is a better way:

                      :find (pull ?e 
                        [* {:block/children [*]}]
                       :where [?e :node/title]]'

One line version: window.roamAlphaAPI.q('[ :find (pull ?e [* {:block/children [*]}]) :where [?e :node/title]]')

This section [* {:block/children [*]}]” translated as pull all components from block ?e, then drop down into each element of children and pull all components from that.

Looking at the output for the same page as above, it is a lot richer:

title: "July 27th, 2020"
uid: "07-27-2020"
children: Array(2)
0: {string: "[[Foo]]", email: "", time: 1595804295662, refs: Array(1), children: Array(2), …}
1: {string: "#[[Quick Capture]]", email: "", time: 1595838215266, refs: Array(1), uid: "RDAWLUY_g", …}

Deeper Still: Recursive Pulls

If you drill down into the content of the above, say by expanding [[Foo]], you notice something annoying. Whilst the content of Foo is populated, the contents of Foo’s children is not available.

title: "July 27th, 2020"
uid: "07-27-2020"
children: Array(4)
    children: Array(2)
        0: {id: 27982}
        1: {id: 27984}
    uid: "d_H0WBRLJ"
    string: "[[Foo]]"
    email: ""
    id: 27983
1: {string: "#[[Quick Capture]]", email: "", time: 1595838215266, refs: Array(1), uid: "RDAWLUY_g", …}

What we need to do is a recursive pull:

                    :find (pull ?e [
                        {:block/children ...}
                    :where [?e :node/title]]')

One line version: window.roamAlphaAPI.q('[ :find (pull ?e [ :node/title :block/string :block/children {:block/children ...} ]) :where [?e :node/title]]')

The clause {:block/children …} will recursively pull all children and their key fields into a javascript graph. Now if you drill down, you will find all children populated.

Query Basics

Let’s try doing some basic queries. Say we want to find all blocks in the database that reference the page called Roam. We could do something like the following:

                        :find (pull ?referencingBlock [*])  
                        :in $ ?pagetitle   
                            [?referencingBlock :block/refs ?referencedPage]
                            [?referencedPage :node/title ?pagetitle]

One line version: window.roamAlphaAPI.q('[ :find (pull ?referencingBlock [*]) :in $ ?pagetitle :where [?referencingBlock :block/refs ?referencedPage] [?referencedPage :node/title ?pagetitle] ]',"Roam")

There is something new in this query: :in $ ?pagetitle - this tells the api that we are going to pass in one argument to the javascript function, called ?pageTitle. In this case this is a string : Roam”

We construct a query by describing different blocks, and their relationship to each other.

[?referencingBlock :block/refs ?referencedPage] : we want to find all blocks that reference referencedPage

[?referencedPage :node/title ?pagetitle] and referencedPage is defined as the block whose title is equal to the argument.

This is a simple example. You could take it further- eg find all children of all blocks that reference a named page.

Reverse Relationships

What if you want to relationships backwards? Say for instance you wanted a list of all pages in the database, with all blocks that referenced those pages? This is possible using the **_**” character, as follows:

                        (pull ?e 
                              [:node/title {:block/_refs [:block/string] }])  
                        [?e :node/title]]')

One line version: window.roamAlphaAPI.q('[ :find (pull ?e [:node/title {:block/_refs [:block/string] }]) :where [?e :node/title]]')

This creates a field” on each page called _refs” which will contain an array of all blocks whose refs” array contain this block. The title of the page and name of the linking block is pulled. This gives something like the following:

5: Array(1)
    title: "July 27th, 2020"
    _refs: Array(2)
        0: {string: "Week Starting [[July 27th, 2020]]"}
        1: {string: "Mon [[July 27th, 2020]]"}


You can also build a relationship between blocks, define it as a relationship, and refer to that relationship within a query. For example:

let ancestorrule='[ 
                   [ (ancestor ?b ?a) 
                        [?a :block/children ?b] ] 
                   [ (ancestor ?b ?a) 
                        [?parent :block/children ?b ] 
                        (ancestor ?parent ?a) ] ] ]';

One line version: let ancestorrule='[ [ (ancestor ?b ?a) [?a :block/children ?b] ] [ (ancestor ?b ?a) [?parent :block/children ?b ] (ancestor ?parent ?a) ] ] ]';

This says:
?a is the ancestor of ?b IF ?b is one of ?as children (lines 2 and 3), OR

?a is the ancestor of ?b IF ?b is one of the children of some other block (called ?parent), AND

?a is the ancestor of ?parent

We can then pass this rule into the q function as follows:

                            (pull ?block [:block/string])  
                        :in $ ?pagetitle % 
                            [?page :node/title ?pagetitle] 
                            (ancestor ?block ?page)

One line version: window.roamAlphaAPI.q('[ :find (pull ?block [:block/string]) :in $ ?pagetitle % :where [?page :node/title ?pagetitle] (ancestor ?block ?page) ]' ,"Roam",ancestorrule);

Note the new variable in the section: :in $ ?pagetitle % . The % variable refers to the rule passed as the second arg in the javascript function.

This returns the contents of all blocks that link to my Roam” page.

Aggregate Functions

Say you wanted to run a variant on the query above which counted the number of outbound references on a page. So you want all descendants of the page block, and then to count the outbound refs from those blocks. You can do this using an aggregate function as follows:

                        :find (count ?refs) 
                        :with ?e :in $ ?myid % 
                            [?e :node/title ?myid]
                            [?b :block/refs ?refs] 
                            (ancestor ?b ?e)]',
                       "Roam", ancestorrule);

One line version: window.roamAlphaAPI.q('[ :find (count ?refs) :with ?e :in $ ?myid % :where [?e :node/title ?myid] [?b :block/refs ?refs] (ancestor ?b ?e)]', "Roam", ancestorrule); This returns an array of arrays, containing the count.

    0: Array(1)
        0: 26

Datomic Reference

For further reading, see the following:

Previous post
RoamPageSearch Quickly jump to and interact with your Roam Research pages from anywhere on your Mac, using Alfred. Download from packal Don’t forget to read the