Infrastructure at your Service

In my previous post, I talked about the Lifecycle of Alfresco Nodes. You may have noticed that I tried to insert in my explanations some elements that are specific to databases (tables, fields, aso…). These elements are quite essential to prepare a post like this one: more database oriented. I already explained what exactly are the consequences on the database side when a node is removed and I will try in this post to share some useful queries regarding these points but not only!

For this post, I used my local Alfresco Community 4.2.c installation with a PostgreSQL database. For your information, it just take 30 minutes to get this test environment ready with the Alfresco’s installer (Windows, Mac or Unix – Not for Production use!). Of course, use the Database only for your daily administration work is certainly not the best idea but in some cases, it can really be faster and easier to just run some SQL commands at the DB level…

 

I. Document information

So let start this post with some generic queries that can be used to retrieve some information about documents. In this part, all columns of the results will be the same because I just pick up the same fields in my queries but the filter part (the WHERE clause) changes a little bit to be able to retrieve some information from different elements.

The first command I would like to show you is how to retrieve some information about documents based on the size of the content. Here, I just uploaded the document “Test_Lifecycle.docx” with a size of 52MB. So based on that, let’s say that I want to retrieve all elements on my Alfresco installation with a content that is bigger than 40MB. In the same approach, you can select all elements with a content that is smaller than or between XX and YYMB. The conversion in MB is done using the round() function. Therefore, if you want this value to be in KB instead, just remove one division by 1024 in each round() function:

  • All documents bigger than 40MB (across all stores so you can have different versions, aso… A filter can be used on the store if needed)
SELECT n.id AS "Node ID",
  n.store_id AS "Store ID",
  concat(s.protocol, '://', s.identifier) AS "Store Name",
  round(u.content_size/1024/1024,2) AS "Size (MB)",
  n.uuid AS "Document ID (UUID)",
  n.audit_creator AS "Creator",
  n.audit_created AS "Creation Date",
  n.audit_modifier AS "Modifier",
  n.audit_modified AS "Modification Date",
  p1.string_value AS "Document Name",
  u.content_url AS "Location"
FROM alf_node AS n,
  alf_node_properties AS p,
  alf_node_properties AS p1,
  alf_namespace AS ns,
  alf_qname AS q,
  alf_content_data AS d,
  alf_content_url AS u,
  alf_store AS s
WHERE n.id=p.node_id
  AND ns.id=q.ns_id
  AND p.qname_id=q.id
  AND p.long_value=d.id
  AND d.content_url_id=u.id
  AND p1.node_id=n.id
  AND p1.qname_id IN (SELECT id FROM alf_qname WHERE local_name='name')
  AND q.local_name='content'
  AND n.store_id=s.id
  AND round(u.content_size/1024/1024,2)>40
ORDER BY u.content_size DESC;

 

I will just put it once but here is the result of this command in my case:

Node ID | Store ID |       Store Name        | Size (MB) |          Document ID (UUID)          | Creator |         Creation Date         | Modifier |       Modification Date       |    Document Name    |                            Location                            
--------+----------+-------------------------+-----------+--------------------------------------+---------+-------------------------------+----------+-------------------------------+---------------------+----------------------------------------------------------------
 131856 |        6 | workspace://SpacesStore |     52.00 | eb267742-c018-4ba5-8ca4-75ca23c860f0 | Morgan  | 2015-04-30T12:05:50.613+02:00 | Morgan   | 2015-04-30T12:05:50.613+02:00 | Test_Lifecycle.docx | store://2015/4/30/12/5/0e111f05-7fcf-4a44-b719-b94cd04dd5ab.bin

 

So why did I selected these fields?!

  • Node ID: can be useful to join different tables
  • Store ID: a Store ID of 6 means that your document is in its active life (in my case, check the query below on alf_store to see the different stores). Still in my case, a Store ID of 5 means that this document has been deleted by a user and is now in the global trashcan
  • Store Name: I added the display of the store so it is easier to know what the Store ID corresponds to. It’s formatted as “PROTOCOL://IDENTIFIER”
  • Size (MB): what we are searching for…
  • Document ID (UUID): the unique identifier of this document. The simplest way to preview this document is just to open the following url in any browser: http://HOSTNAME:PORT/share/page/document-details?nodeRef=PROTOCOL://IDENTIFIER/eb267742-c018-4ba5-8ca4-75ca23c860f0 (workspace://SpacesStore for store_id=6)
  • Creator, Modifier, Dates: well…
  • Document Name: can be useful to know the type of document without opening an URL (file extension)
  • Location: the actual location of the content’s file on the File System. The “store://” refers to $ALF_DATA/contentstore/

The second command I would like to show you is how to retrieve some information based on the actual UUID of a document. As explained above, the UUID of a document can be found in the URL of its detail’s page:

  • A document using its UUID
SELECT n.id AS "Node ID",
  n.store_id AS "Store ID",
  round(u.content_size/1024/1024,2) AS "Size (MB)",
  n.uuid AS "Document ID (UUID)",
  n.audit_creator AS "Creator",
  n.audit_created AS "Creation Date",
  n.audit_modifier AS "Modifier",
  n.audit_modified AS "Modification Date",
  p1.string_value AS "Document Name",
  u.content_url AS "Location"
FROM alf_node AS n,
  alf_node_properties AS p,
  alf_node_properties AS p1,
  alf_namespace AS ns,
  alf_qname AS q,
  alf_content_data AS d,
  alf_content_url AS u
WHERE n.id=p.node_id
  AND ns.id=q.ns_id
  AND p.qname_id=q.id
  AND p.long_value=d.id
  AND d.content_url_id=u.id
  AND p1.node_id=n.id
  AND p1.qname_id IN (SELECT id FROM alf_qname WHERE local_name='name')
  AND n.uuid='eb267742-c018-4ba5-8ca4-75ca23c860f0';

 

Another possible command would be to find some information based on the File System location. That can be useful for example if there is a big document on the File System and you want to know the type of this document with the extension, its name or maybe some other information about the creator/modifier:

  • A document using its path on the File System
SELECT n.id AS "Node ID",
  n.store_id AS "Store ID",
  round(u.content_size/1024/1024,2) AS "Size (MB)",
  n.uuid AS "Document ID (UUID)",
  n.audit_creator AS "Creator",
  n.audit_created AS "Creation Date",
  n.audit_modifier AS "Modifier",
  n.audit_modified AS "Modification Date",
  p1.string_value AS "Document Name",
  u.content_url AS "Location"
FROM alf_node AS n,
  alf_node_properties AS p,
  alf_node_properties AS p1,
  alf_namespace AS ns,
  alf_qname AS q,
  alf_content_data AS d,
  alf_content_url AS u
WHERE n.id=p.node_id
  AND ns.id=q.ns_id
  AND p.qname_id=q.id
  AND p.long_value=d.id
  AND d.content_url_id=u.id
  AND p1.node_id=n.id
  AND p1.qname_id IN (SELECT id FROM alf_qname WHERE local_name='name')
  AND u.content_url='store://2015/4/30/12/5/0e111f05-7fcf-4a44-b719-b94cd04dd5ab.bin';

 

II. Number of…

From a reporting point of view, let’s say that you need some information regarding the number of… something. In this case and if you want to use your DB directly, then there is a really simple solution (simple but is it the best?) because Alfresco provide a Database architecture that is quite simple to understand and to use to get what you need. Indeed, if you take a look at the “alf_qname” table, you will see that every element that is part of Alfresco has its QName listed here. A QName is the Qualified Name of a repository item. This can be seen as a kind of “Super-Type”:

alfresco=> SELECT * FROM alf_qname;
 id | version | ns_id | local_name
----+---------+-------+------------
  1 |       0 |     1 | store_root
  2 |       0 |     1 | aspect_root
  3 |       0 |     1 | container
  4 |       0 |     1 | children
  5 |       0 |     2 | user
...
 24 |       0 |     6 | folder
...
 51 |       0 |     6 | content
...
133 |       0 |     6 | thumbnail
134 |       0 |    13 | rendition
...

 

As you can see above, if you are searching for something that has a content, it can be done quite easily using the id or the local_name that correspond to that. So based on this table, here are some queries that can be useful:

  • Retrieve the number of users in the Repository
SELECT count(*)
FROM alf_node AS n,
  alf_qname AS q
WHERE n.type_qname_id=q.id
  AND q.local_name='user';

 

  • Retrieve the number of elements with a content in the Repository (include system’s documents)
SELECT count(*)
FROM alf_node AS n,
  alf_qname AS q
WHERE n.type_qname_id=q.id
  AND q.local_name='content';

 

  • Retrieve the number of thumbnails in the Repository
SELECT count(*)
FROM alf_node AS n,
  alf_qname AS q
WHERE n.type_qname_id=q.id
  AND q.local_name='thumbnail';

 

  • Retrieve the number of renditions in the Repository
SELECT count(*)
FROM alf_node AS n,
  alf_qname AS q
WHERE n.type_qname_id=q.id
  AND q.local_name='rendition';

 

Of course you can do that for all QNames but you can also be more precise! So based on the query to retrieve the number of elements with a content, if you only want the number of documents of a specific type, then you can simply complete your query (this suppose the documents kept their extension… which should be the case):

  • Retrieve the number of XML documents in the Repository
SELECT count(*)
FROM alf_node AS n,
  alf_qname AS q,
  alf_node_properties AS p
WHERE n.type_qname_id=q.id
  AND p.node_id=n.id
  AND p.qname_id IN (SELECT id FROM alf_qname WHERE local_name='name')
  AND q.local_name='content'
  AND p.string_value LIKE '%.xml';

 

  • Retrieve the number of PDF documents in the Repository
SELECT count(*)
FROM alf_node AS n,
  alf_qname AS q,
  alf_node_properties AS p
WHERE n.type_qname_id=q.id
  AND p.node_id=n.id
  AND p.qname_id IN (SELECT id FROM alf_qname WHERE local_name='name')
  AND q.local_name='content'
  AND p.string_value LIKE '%.pdf';

 

As the creation date, creator, modification date and modifier information are also stored on the “alf_node” table, you can also very easily filter your query based on the creation/update date of an Alfresco Node. That’s pretty cool, right?! ;)

 

III. Lifecycle specific

To complete the relation between this blog post and the previous one, I wanted to share some queries that can be used to identify the current state of a document. As explained in my previous post, a document that is not yet deleted will be in the store named “workspace://SpacesStore”. A document that has been deleted by a user will be in the sotre named “archive://SpacesStore” and when this document is removed from the global trashcan, the orphan_time is set to the current timestamp. With all these information and with the “alf_node” and “alf_content_url” tables we can easily build our own queries to find what is needed.

alfresco=> SELECT * FROM alf_store;
 id | version | protocol  |       identifier        | root_node_id
----+---------+-----------+-------------------------+--------------
  1 |       1 | user      | alfrescoUserStore       |            1
  2 |       1 | system    | system                  |            5
  3 |       1 | workspace | lightWeightVersionStore |            9
  4 |       1 | workspace | version2Store           |           10
  5 |       1 | archive   | SpacesStore             |           11
  6 |       1 | workspace | SpacesStore             |           12
(6 rows)

 

So let’s find all documents that have been created and aren’t deleted yet. In the queries below, you can replace the store_id= and type_qname_id= with some SELECT queries as I showed you previously in this blog. Since there are already some examples, I will use the IDs directly:

  • All documents created in their active life
SELECT *
FROM alf_node
WHERE store_id=6
  AND type_qname_id=51;

 

The next step on the lifecycle is when the documents have been deleted by a user but aren’t deleted from the global trashcan (orphan_time is still NULL):

  • All documents created that are in the global trashcan (deleted by users)
SELECT *
FROM alf_node
WHERE store_id=5
  AND type_qname_id=51;

 

Finally, when the documents are removed from the global trashcan, some references/fields are removed, the QName of these documents change from “content” (51) to “deleted” (140) on the “alf_node” table and the orphan_time is set to the current timestamp on the “alf_content_url” table:

  • All elements that have been removed from the global trashcan and that are now orphaned
SELECT *
FROM alf_content_url
WHERE orphan_time IS NOT NULL;

 

IV. Other examples

This section of the blog has been added starting in 2017. There were several comments on this blog asking for help on how to get information about several things so I thought I would just add a new section and complete the blog with new queries that can be useful for some people. So here we go:

  • Listing all existing Sites with names, descriptions and visibility
SELECT n.id AS "Node ID",
  n.store_id AS "Store ID",
  n.uuid AS "Site ID (UUID)",
  p.string_value AS "Site Name",
  p1.string_value AS "Site Description",
  p2.string_value AS "Site Visibility",
  n.audit_creator AS "Creator",
  n.audit_created AS "Creation Date",
  n.audit_modifier AS "Modifier",
  n.audit_modified AS "Modification Date"
FROM alf_node AS n,
  alf_node_properties AS p,
  alf_node_properties AS p1,
  alf_node_properties AS p2,
  alf_qname AS q
WHERE n.type_qname_id=q.id
  AND p.node_id=n.id
  AND p1.node_id=n.id
  AND p2.node_id=n.id
  AND p.qname_id IN (SELECT id FROM alf_qname WHERE local_name='name')
  AND p1.qname_id IN (SELECT id FROM alf_qname WHERE local_name='description')
  AND p2.qname_id IN (SELECT id FROM alf_qname WHERE local_name='siteVisibility')
  AND q.local_name='site';

 

  • Size and number of documents created by all users
SELECT n.audit_creator AS "Creator",
  round(sum(s.bin_size)/1024/1024,2) AS "Total Size (MB)",
  count(*) AS "Nb Document"
FROM alf_node AS n,
  alf_qname AS q,
  (SELECT n1.id AS "id",
     u1.content_size AS "bin_size"
   FROM alf_node AS n1,
     alf_node_properties AS p1,
     alf_content_data AS d1,
     alf_content_url AS u1
   WHERE n1.id=p1.node_id
     AND p1.long_value=d1.id
     AND d1.content_url_id=u1.id
     AND p1.qname_id=(SELECT id FROM alf_qname WHERE local_name='content')) AS s
WHERE n.type_qname_id=q.id
  AND n.id=s.id
  AND q.local_name='content'
GROUP BY n.audit_creator
ORDER BY sum(s.bin_size) DESC;

 

  • Retrieve all users with their username and status:
SELECT n.id,
  n.uuid,
  q.local_name AS "Type",
  p1.string_value AS "Username",
  p2.boolean_value AS "Enabled"
FROM alf_node AS n,
  alf_qname AS q,
  alf_node_properties AS p1,
  alf_node_properties AS p2
WHERE n.type_qname_id=q.id
  AND n.id=p1.node_id
  AND p1.node_id=p2.node_id
  AND q.local_name='user'
  AND p1.string_value!=''
  AND p1.qname_id IN (SELECT id FROM alf_qname WHERE local_name='username')
  AND p2.qname_id IN (SELECT id FROM alf_qname WHERE local_name='enabled');

 

  • Disabling all users (except ‘admin’). Please note that you really (REALLY) shouldn’t do this on the DB as it skips all caches, aso… You would need to restart Alfresco for that and it would only be for the Alfresco inline users, it wouldn’t prevent them to login using an AD, if configured. This should really be done from the API layer instead.
UPDATE alf_node_properties
  SET boolean_value=false
WHERE node_id IN (SELECT n1.id
    FROM alf_node AS n1,
      alf_node_properties AS p1
    WHERE n1.type_qname_id IN (SELECT q1.id FROM alf_qname AS q1 WHERE q1.local_name='user')
      AND n1.id=p1.node_id
      AND p1.qname_id IN (SELECT q2.id FROM alf_qname AS q2 WHERE q2.local_name='username')
      AND p1.string_value NOT IN ('admin'))
 AND qname_id IN (SELECT q3.id
    FROM alf_qname AS q3
    WHERE q3.local_name='enabled');

 

For the next commands, let’s use the document that I created in my previous blog regarding the lifecycle and that I mentioned at the beginning of this blog. The Document name was “Test_Lifecycle.docx” and its node_id was “131856”. With the very first SQL from this blog, I could find its location on the file system… But what about its location on Alfresco?

  • Retrieve the parent node_id of an Alfresco node from the node’s name (files, folders, aso…) – Obviously the name needs to be unique otherwise you will see several parents node_id
SELECT child_node_id AS "Parent node_id",
  qname_localname AS "Parent name"
FROM alf_child_assoc
WHERE type_qname_id IN (SELECT id FROM alf_qname WHERE local_name='contains')
  AND child_node_id IN (SELECT parent_node_id
    FROM alf_child_assoc
    WHERE qname_localname='Test_Lifecycle.docx');

 

  • Retrieve the Parent node_id of an Alfresco node from the child node’s id (files, folders, aso…)
SELECT child_node_id AS "Parent node_id",
  qname_localname AS "Parent name"
FROM alf_child_assoc
WHERE type_qname_id IN (SELECT id FROM alf_qname WHERE local_name='contains')
  AND child_node_id IN (SELECT parent_node_id
    FROM alf_child_assoc
    WHERE child_node_id='131856');

 

From this point, you could obviously continue to go up & up again on the folder’s hierarchy by just changing the child_node_id with the Parent node_id and you would at some point end-up at the root of the Alfresco tree: the “company_home”. Now if you don’t want to do it manually, I also have something for you:

  • Recursively find the parents of an Alfresco node (from its id, uniqueness is important here) – 1 node per line
WITH RECURSIVE parents AS
(SELECT a1.id,
    a1.child_node_id,
    a1.qname_localname AS "child_name",
    a1.parent_node_id
  FROM alf_child_assoc AS a1
  WHERE a1.child_node_id='131856'
  UNION
  SELECT a2.id,
    a2.child_node_id,
    a2.qname_localname AS "child_name",
    a2.parent_node_id
  FROM alf_child_assoc AS a2
  JOIN parents p ON a2.child_node_id=p.parent_node_id
)
SELECT * FROM parents;

 

If the file “Test_Lifecycle.docx” is present in Alfresco under a folder named “MyFolder” in a site name “MySite”, then this would be a possible outcome of the query:

   id   | child_node_id |      child_name      | parent_node_id
--------+---------------+----------------------+----------------
 956325 |        131856 | Test_Lifecycle.docx  |         131800
 957208 |        131800 | MyFolder             |         131732
 943025 |        131732 | documentLibrary      |         131704
 943002 |        131704 | MySite               |         130207
  72781 |        130207 | sites                |             13
      7 |            13 | company_home         |             12
 (6 rows)

 

If you rather prefer to display the full path directly, I also have something for you!

  • Recursively find the parents of an Alfresco node (from its id, uniqueness is important here) – full path
WITH RECURSIVE parents AS
(SELECT a1.id,
    a1.child_node_id,
    a1.qname_localname AS "child_name",
    a1.parent_node_id
  FROM alf_child_assoc AS a1
  WHERE a1.child_node_id='131856'
  UNION
  SELECT a2.id,
    a2.child_node_id,
    a2.qname_localname AS "child_name",
    a2.parent_node_id
  FROM alf_child_assoc AS a2
  JOIN parents p ON a2.child_node_id=p.parent_node_id
)
SELECT string_agg(child_name,' > ') AS "file_path"
FROM (SELECT *,
    ROW_NUMBER () OVER ()
  FROM parents
  ORDER BY ROW_NUMBER DESC)
AS ordered_parents;

 

This time the outcome would be the following one:

                                    file_path
----------------------------------------------------------------------------------
 company_home > sites > MySite > documentLibrary > MyFolder > Test_Lifecycle.docx
 (1 row)

 

Then you could also do it the other way around (up->down instead of down->up). So starting with a parent and retrieving all its childs. There are some things you need to be aware of first: all the Alfresco nodes will be retrieved and not just folders/files! You can put a restriction on the “contains” alf_qname like I did in a few queries above but you will still have some hidden files and stuff like that, especially for the sites (surf-config, user’s dashboards setup, aso…)

  • Recursively find the number of childs (Alfresco nodes) of a specific Alfresco node (included in the count – in the example below, I’m using “company_home” = 13 (don’t run that on Billions of nodes!))
WITH RECURSIVE childs AS
(SELECT a1.id,
    a1.child_node_id,
    a1.qname_localname AS "child_name",
    a1.parent_node_id,
    a1.type_qname_id
  FROM alf_child_assoc AS a1
  WHERE a1.child_node_id='13'
  UNION
  SELECT a2.id,
    a2.child_node_id,
    a2.qname_localname AS "child_name",
    a2.parent_node_id,
    a2.type_qname_id
  FROM alf_child_assoc AS a2
  JOIN childs c ON a2.parent_node_id=c.child_node_id
)
SELECT count(*)
FROM childs
WHERE type_qname_id IN (SELECT id
  FROM alf_qname
  WHERE local_name='contains');

 

  • Listing the permissions assigned to a specific Alfresco node (it’s possible to use n.id instead of n.uuid if you have the Node ID)
SELECT n.id AS "Node ID",
  p.string_value AS "Document Name",
  ap.id AS "Permission ID",
  ap.name AS "Permission Name",
  aa.authority AS "User/Group",
  aace.allowed AS "Allowed",
  aacl.inherits AS "Inheritance Enabled",
  CASE WHEN aam.pos=0 THEN 'f'
    ELSE 't'
  END AS "Inherited"
FROM alf_node AS n,
  alf_node_properties AS p,
  alf_acl_member AS aam,
  alf_access_control_entry AS aace,
  alf_authority as aa,
  alf_access_control_list AS aacl,
  alf_permission AS ap
WHERE n.id=p.node_id
  AND n.acl_id=aam.acl_id
  AND aam.ace_id=aace.id
  AND aa.id=aace.authority_id
  AND aacl.id=aam.acl_id
  AND ap.id=aace.permission_id
  AND p.qname_id IN (SELECT id FROM alf_qname WHERE local_name='name')
  AND n.uuid='eb267742-c018-4ba5-8ca4-75ca23c860f0';

 

The “Inheritance Enabled” column is whether or not the inheritance is enabled on this specific node while the “Inherited” column is whether or not this specific permission has been set on the node locally (value = f) or is present here because of the inheritance (value = t) in which case the permission has been set on a parent (direct or not). The outcome would be something like that:

Node ID |    Document Name    | Permission ID | Permission Name |    User/Group     | Allowed | Inheritance Enabled | Inherited
--------+---------------------+---------------+-----------------+-------------------+---------+---------------------+-----------
 131856 | Test_Lifecycle.docx |            21 | dbi_Security    | GROUP_Accounting  | t       | t                   | t
 131856 | Test_Lifecycle.docx |            20 | dbi_Delete      | GROUP_Management  | t       | t                   | t
 131856 | Test_Lifecycle.docx |            18 | dbi_Read        | GROUP_Consultants | t       | t                   | t
 131856 | Test_Lifecycle.docx |            19 | dbi_Write       | GROUP_EVERYONE    | t       | t                   | f
 131856 | Test_Lifecycle.docx |            18 | dbi_Read        | MorganPatou       | t       | t                   | f
(5 rows)

 

These are a few examples… If you have some that you think should be shared, don’t hesitate to post them in the comments!

I hope you enjoyed this blog post because it was quite hard for me to write something about database queries without giving up my soul to the DB world! See you soon ;).

Morgan Patou
Morgan Patou

Senior Consultant