Directory_Browsing#

Overview#

The IPA Web UI needs to provide a way to browse the entire content of IPA directory (see ticket #981). The current mechanism has some issues:

  • The UI doesn’t show the entire directory. Due to DS restrictions on plain LDAP search, currently the UI will only show the first 100 entries. Simply increasing or removing the limit would be impractical because the UI may have to handle a large amount of data and also it might degrade server performance.

  • The UI doesn’t provide a way to sort the entries. The entries needs to be sorted based on the entire directory, not based on the entries currently displayed in the UI.

There are several options to address these issues:

  1. Using Simple Paged Results to retrieve entries

  2. Using Simple Paged Results to retrieve primary keys

  3. Virtual List View

  4. Hybrid solution

Option #1: Using Simple Paged Results to retrieve entries#

Simple Paged Results allows the UI to retrieve all entries in multiple pages. It also supports server-side sorting. The problem is that the pages are returned sequentially, so the UI will only able to provide a ‘Next’ button. It will not be able to go to a previous or specific page without re-searching from the beginning. This option also requires the server to maintain an open connection to the DS while the entries are being retrieved (see ticket #215).

This is how it will work:

  1. The admin opens the first page of the Users list page in the UI.

  2. The UI issues ipa user-find –offset=0 –count=100 –orderby=uid,cn,email.

  3. IPA server creates a session and uses simple paged results to retrieve the first 100 from the DS.

  4. IPA server returns the results to the UI but it keeps the DS connection open.

  5. The UI shows the results of the first page.

  6. The admin clicks Next.

  7. The UI issues ipa user-find –offset=100 –count=100.

  8. IPA server users the same connection to retrieve the next 100 users.

  9. The UI shows the results of the second page.

Option #2: Using Simple Paged Results to retrieve primary keys#

The UI could also use Simple Paged Results to retrieve all primary keys (see ticket #1262). Since it has the full list, the UI can go to any page directly. The UI can then retrieve the additional data (e.g. givenName, sn) for the entries that are going to be displayed in the current page. This option does not require server session.

This is how it will work:

  1. The admin opens the first page of the Users list page in the UI.

  2. The UI issues ipa user-find –primary-keys –orderby=uid,cn,email.

  3. The server uses simple paged results to retrieve all primary keys from the DS.

  4. The UI constructs a batch command to retrieve the attributes of the first 100 users. The batch command consists of ipa user-show operations.

  5. The server executes the batch command and return the results to the UI.

  6. The UI shows the results of the first page.

  7. The admin opens a different page (it doesn’t have to be sequential).

  8. The UI calculates the offset, then constructs a batch command to retrieve the attributes of the users in that page.

  9. The server executes the batch command and return the results to the UI.

  10. The UI shows the results of that page.

A complete list of primary keys will be smaller than a complete list of entries with the attributes. However, for a large directory it might still be a problem.

Option #3: Virtual List View (VLV)#

VLV allows IPA server to retrieve any page of the search result directly. The UI will only need to send the offset and the size for the current page. The problem is that the VLV index has to be created ahead of time and it has a fixed base, scope, filter, and sort order. Only search requests that match this configuration will benefit from VLV. This option does not require server session.

This is how it will work:

  1. The admin opens the first page of the Users list page in the UI.

  2. The UI issues ipa user-find –offset=0 –count=100 –orderby=uid,cn,email.

  3. IPA server uses VLV to retrieve the users from the DS based on the requested offset and count.

  4. IPA server returns the results to the UI.

  5. The UI shows the results of the first page.

  6. The admin opens a different page (it doesn’t have to be sequential).

  7. The UI calculates the offset and issues ipa user-find command.

  8. IPA server uses VLV to retrieve the users from the DS based on the requested offset and count.

  9. IPA server returns the results to the UI.

  10. The UI shows the results of that page.

Option #4: Hybrid solution#

The DS can prepare a few standard VLV indexes, for example “Users sorted by UID”, “Users sorted by email”. The UI can use them to handle the most common use cases: browsing without filter and sorted by one attribute.

For less common use cases, the UI can use one of the earlier solutions using Simple Paged Results.

When the admin opens the list page, the UI determines which type of operation it’s going to execute.

Configuring VLV#

Configure DS:

ldapadd -x -D "cn=Directory Manager" -w Secret123 << EOF
dn: cn=Users cn=users cn=accounts dc=example dc=com,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
objectClass: top
objectClass: vlvSearch
cn: Users cn=users cn=accounts dc=example dc=com
vlvBase: cn=users,cn=accounts,dc=example,dc=com
vlvScope: 1
vlvFilter: (objectclass=*)

dn: cn=by UID cn=users cn=accounts dc=example dc=com,cn=Users cn=users cn=accounts dc=example dc=com,
  cn=userRoot,cn=ldbm database,cn=plugins,cn=config
objectClass: top
objectClass: vlvIndex
cn: by UID cn=users cn=accounts dc=example dc=com
vlvSort: uid givenName sn
EOF

Stop DS:

service dirsrv stop EXAMPLE-COM

Generate indexes:

/var/lib/dirsrv/scripts-EXAMPLE-COM/vlvindex -n userRoot \
-T "by UID cn=users cn=accounts dc=example dc=com"

Start DS:

service dirsrv start EXAMPLE-COM

Using VLV#

ldapsearch -x -D "cn=Directory Manager" -w Secret123 -b "cn=users,cn=accounts,dc=example,dc=com" -s one \
-E \!vlv=0/100/1/0 -E \!sss=uid/givenName/sn \
"(objectclass=*)" dn givenName sn

Using Simple Paged Results#

ldapsearch -x -D "cn=Directory Manager" -w Secret123 -b "cn=users,cn=accounts,dc=example,dc=com" -s one \
-E \!pr=100 -E \!sss=uid/givenName/sn \
"(objectclass=*)" dn givenName sn

References#