Introduction to the Roam Alpha API
RoamAlphaAPI is a javascript API available within Roam that allows querying of a Roam database using Datomic/Datalog.
This page is a summary of what I learned writing RoamPageSearch. I am not a clojure/datalog expert, and my javascript is pretty iffy, but all the examples work so hopefully this might be of use to some people.
Roam Alpha API exposes two datomic functions: q - allows querying of the Roam graph. pull- allows pulling of details of one block
Getting started with Roam Alpha API
The easiest way to get started is to try some examples in your browser developer console, after bringing up the Roam page in your browser. In Chrome, hit ⌘⌥+ I to bring up the developer tools , choose the console tab, and paste some of the examples below at the prompt “>”.
You can navigate the results of the queries in the console by clicking on them to expand, or you can write javascript code to do interesting stuff with the results.
Each of the examples below can be copied and pasted into your Roam console. Note that the examples must be on one line.
A word of warning: if setting variables in the console, avoid short variable names as they may collide with variables Roam uses. If this does happen during your travels, just refresh the page.
You can paste the examples below into the console, however note that you need to use the version without line breaks.
Finding all pages
Find all blocks with a :node/title element (ie all pages), and return their database ids. window.roamAlphaAPI.q('[:find ?e :where [?e :node/title] ]')
In this example,
[?e :node/title] matches “any block with a field called :node/title. ?e is a variable referring to the block that matches
The example returns an array of arrays- the outer array has an entry for each page, and the inner array has one element which is the database id of the page.
Getting Details of One Block
If you know the database id of one block, you can query all details of that block. For example, my Roam page is block 4 in my database. Running the following: window.roamAlphaAPI.pull('[*]',4) returns a javascript object similar to the following:
:node/title: "Roam"
:block/uid: "MOa3Jbhca"
:attrs/lookup: Array(3)
0: {:db/id: 4}
1: {:db/id: 2636}
2: {:db/id: 11268}
:: Array(16)
0: {:db/id: 2471}
1: {:db/id: 3473}
2: {:db/id: 3526}
3: {:db/id: 3532}
4: {:db/id: 3550}
5: {:db/id: 11267}
6: {:db/id: 11268}
7: {:db/id: 23920}
8: {:db/id: 23921}
9: {:db/id: 23923}
10: {:db/id: 26049}
11: {:db/id: 26050}
12: {:db/id: 26054}
13: {:db/id: 30550}
14: {:db/id: 30551}
15: {:db/id: 30552}
:block/open: true
:children/view-type: ":bullet"
:create/email: "foo@bar.com"
:create/time: 1580480550337
:db/id: 4
:edit/email: "foo@bar.com"
:edit/time: 1597092215174
:entity/attrs: [Array(3)]It’s worth highlighting the key fields and types of blocks. All blocks have the following :db/id- integer id representing id in database
:block/uid - 9 char string representing block uid, eg “MOa3Jbhca” - Prepending this with “https://roamresearch.com/#/app/YOURDBNAME/page/” will give a link to that page or block.
Blocks may have the following:
:block/children: which returns an array of blocks that are children of the target.
Pages have a special field: :node/title: which contains the page name
whereas all other blocks have a field called:
:block/string: which contains the block content as a string.
:block/refs: If a block references other blocks, then it may have this field which contains those blocks reference.
Putting “q” and “pull” together : the stupid way
So if you wanted a full representation of all pages in your database, you could run the following:
window.roamAlphaAPI.q('[
:find ?e
:where [?e :node/title] ]
')
.map(page=>
window.roamAlphaAPI.pull('[*]',page[0]
))One line version: window.roamAlphaAPI.q('[ :find ?e :where [?e :node/title] ] ') .map(page=> window.roamAlphaAPI.pull('[*]',page[0]))
The “q” part returns an array of pages reference, each of which is fed into a “pull” expression to pull all fields. This returns an array of javascript object of pages, that we can do fun stuff with.
Putting “q” and “pull” together : Using Pull Parameters
However there is a better way of doing the above.
window.roamAlphaAPI.q('[
:find (pull ?e [*])
:where [?e :node/title]]
')One line version: window.roamAlphaAPI.q('[ :find (pull ?e [*]) :where [?e :node/title]] ')
This uses a pull expression “(pull ?e [*])”” which is invoked for each node, and pulls all fields defined as “component fields”, which are whatever Roam has defined as fields to be pulled on each query.
Going Deeper: Pulling Children
If we drill into the output of the command above, we find something annoying. When a block points to other blocks (eg :block/children or :block/refs), we only have access to the dbid of those other blocks. For example, here is the object returned from the q/pull above showing one of my daily pages:
title: "July 27th, 2020"
uid: "07-27-2020"
children: Array(2)
0: {id: 27983}
1: {id: 28054}What if we want to get the content of those children? We can always go back and do pulls for each block, but there is a better way:
window.roamAlphaAPI.q('[
:find (pull ?e
[* {:block/children [*]}]
)
:where [?e :node/title]]'
)One line version: window.roamAlphaAPI.q('[ :find (pull ?e [* {:block/children [*]}]) :where [?e :node/title]]')
This section “[* {:block/children [*]}]” translated as “pull all components from block ?e, then drop down into each element of children and pull all components from that.
Looking at the output for the same page as above, it is a lot richer:
title: "July 27th, 2020"
uid: "07-27-2020"
children: Array(2)
0: {string: "[[Foo]]", email: "foo@bar.com", time: 1595804295662, refs: Array(1), children: Array(2), …}
1: {string: "#[[Quick Capture]]", email: "foo@bar.com", time: 1595838215266, refs: Array(1), uid: "RDAWLUY_g", …}Deeper Still: Recursive Pulls
If you drill down into the content of the above, say by expanding [[Foo]], you notice something annoying. Whilst the content of Foo is populated, the contents of Foo’s children is not available.
title: "July 27th, 2020"
uid: "07-27-2020"
children: Array(4)
0:
children: Array(2)
0: {id: 27982}
1: {id: 27984}
uid: "d_H0WBRLJ"
string: "[[Foo]]"
email: "foo@bar.com"
id: 27983
1: {string: "#[[Quick Capture]]", email: "foo@bar.com", time: 1595838215266, refs: Array(1), uid: "RDAWLUY_g", …}What we need to do is a recursive pull:
window.roamAlphaAPI.q('[
:find (pull ?e [
:node/title
:block/string
:block/children
{:block/children ...}
])
:where [?e :node/title]]')One line version: window.roamAlphaAPI.q('[ :find (pull ?e [ :node/title :block/string :block/children {:block/children ...} ]) :where [?e :node/title]]')
The clause ” {:block/children …}” will recursively pull all children and their key fields into a javascript graph. Now if you drill down, you will find all children populated.
Query Basics
Let’s try doing some basic queries. Say we want to find all blocks in the database that reference the page called Roam. We could do something like the following:
window.roamAlphaAPI.q('[
:find (pull ?referencingBlock [*])
:in $ ?pagetitle
:where
[?referencingBlock :block/refs ?referencedPage]
[?referencedPage :node/title ?pagetitle]
]',"Roam")One line version: window.roamAlphaAPI.q('[ :find (pull ?referencingBlock [*]) :in $ ?pagetitle :where [?referencingBlock :block/refs ?referencedPage] [?referencedPage :node/title ?pagetitle] ]',"Roam")
There is something new in this query: :in $ ?pagetitle - this tells the api that we are going to pass in one argument to the javascript function, called ?pageTitle. In this case this is a string : “Roam”
We construct a query by describing different blocks, and their relationship to each other.
[?referencingBlock :block/refs ?referencedPage] : we want to find all blocks that reference “referencedPage”
[?referencedPage :node/title ?pagetitle] and referencedPage is defined as the block whose title is equal to the argument.
This is a simple example. You could take it further- eg find all children of all blocks that reference a named page.
Reverse Relationships
What if you want to relationships backwards? Say for instance you wanted a list of all pages in the database, with all blocks that referenced those pages? This is possible using the “**_**” character, as follows:
window.roamAlphaAPI.q('[
:find
(pull ?e
[:node/title {:block/_refs [:block/string] }])
:where
[?e :node/title]]')One line version: window.roamAlphaAPI.q('[ :find (pull ?e [:node/title {:block/_refs [:block/string] }]) :where [?e :node/title]]')
This creates a “field” on each page called “_refs” which will contain an array of all blocks whose “refs” array contain this block. The title of the page and name of the linking block is pulled. This gives something like the following:
5: Array(1)
0:
title: "July 27th, 2020"
_refs: Array(2)
0: {string: "Week Starting [[July 27th, 2020]]"}
1: {string: "Mon [[July 27th, 2020]]"}Rules
You can also build a relationship between blocks, define it as a relationship, and refer to that relationship within a query. For example:
let ancestorrule='[
[ (ancestor ?b ?a)
[?a :block/children ?b] ]
[ (ancestor ?b ?a)
[?parent :block/children ?b ]
(ancestor ?parent ?a) ] ] ]';One line version: let ancestorrule='[ [ (ancestor ?b ?a) [?a :block/children ?b] ] [ (ancestor ?b ?a) [?parent :block/children ?b ] (ancestor ?parent ?a) ] ] ]';
This says:
?a is the ancestor of ?b IF ?b is one of ?as children (lines 2 and 3), OR
?a is the ancestor of ?b IF ?b is one of the children of some other block (called ?parent), AND
?a is the ancestor of ?parent
We can then pass this rule into the q function as follows:
window.roamAlphaAPI.q('[
:find
(pull ?block [:block/string])
:in $ ?pagetitle %
:where
[?page :node/title ?pagetitle]
(ancestor ?block ?page)
]'
,"Roam",ancestorrule);One line version: window.roamAlphaAPI.q('[ :find (pull ?block [:block/string]) :in $ ?pagetitle % :where [?page :node/title ?pagetitle] (ancestor ?block ?page) ]' ,"Roam",ancestorrule);
Note the new variable in the section: :in $ ?pagetitle % . The % variable refers to the rule passed as the second arg in the javascript function.
This returns the contents of all blocks that link to my “Roam” page.
Aggregate Functions
Say you wanted to run a variant on the query above which counted the number of outbound references on a page. So you want all descendants of the page block, and then to count the outbound refs from those blocks. You can do this using an aggregate function as follows:
window.roamAlphaAPI.q('[
:find (count ?refs)
:with ?e :in $ ?myid %
:where
[?e :node/title ?myid]
[?b :block/refs ?refs]
(ancestor ?b ?e)]',
"Roam", ancestorrule);One line version: window.roamAlphaAPI.q('[ :find (count ?refs) :with ?e :in $ ?myid % :where [?e :node/title ?myid] [?b :block/refs ?refs] (ancestor ?b ?e)]', "Roam", ancestorrule); This returns an array of arrays, containing the count.
[Array(1)]
0: Array(1)
0: 26Datomic Reference
For further reading, see the following: https://docs.datomic.com/on-prem/pull.html https://docs.datomic.com/on-prem/query.html https://docs.datomic.com/cloud/query/query-data-reference.html