107. Creating Filters with ListObjects

Learning goal: Understand ListObjects and how they can be used to produce filters. The code for this chapter can be found here.

Defining ListObjects

With our model open, we can start to interact with the data by creating Generic Objects. Generic Objects have dynamic properties that can be defined for working with dynamic data from the Engine. There are several dynamic property types that can be used by the Engine. They are detailed in this list.

In our dashboard design, we have two filters on the left side of our page. Each of these filters corresponds with a single field that our user can explore and filter in. The dynamic structure ListObject is best suited for building this functionality. According to the API documentation,

“A list object is a visualization that contains one dimension. When the engine calculates the layout of a list object, it calculates all values. If some selections are applied, the selected values are displayed along with the excluded and optional values.”

In other words, a list object:

  • lists all the values in a single field

  • provides metadata about the current state of each field value (selected, possible, etc)

  • has methods for searching and filtering the field

To create a ListObject, we create a dynamic property for it in a Generic Object. Remember, Generic Objects are defined with JavaScript objects. These objects are defined with static and dynamic properties. All Generic Objects must have a property called qInfo that defines a qType. This is the only mandatory property. The most basic Generic Object might be structured like the example to the right.

When we add properties to this Generic Object, how does the Engine know which properties are dynamic? The Engine parses dynamic properties based on the name of the property. If we want a list object, we pass it a JSON definition for a list object, known as a ListObjectDef, via a property named qListObjectDef. For any property in our Generic Object definition that is named qListObjectDef, the Engine will know to parse that property’s value as a ListObjectDef to produce a ListObject.

The basic Generic Object structure demonstrated before can be modified to add a ListObject like so:

When these properties are evaluated with layout call, the resulting layout will translate the ListObjectDef into a ListObject and the layout will have the following structure:

In general, when you see property names prefaced by “q”, those are properties that are evaluated or produced by the Engine.

Let’s explore how you write a ListObjectDef by creating a Generic Object for our Region filter. To keep things clean, we will separate our code into different modules. We can store our Generic Object definition as a JSON file. Create a JSON file called “region-listobject.json” under a new directory called “defs” within the “src” folder. The final folder path should be src/defs/region-listobject.json.

In this file, let’s start by creating a basic Generic Object with no ListObject defined:

The “qType” property here can be anything you want; types are useful if you need to manage lots of objects in a model. For our simple dashboard, this is not a concern.

Before we add a ListObjectDef, let’s load this definition into our index.js file and create a session object with it. To load the file in, let’s add a require statement to the top of our code. We can also add comments to make it clearer what the code is doing:

Next, we need to create a Generic Object in our model with this definition. The App class has two methods for creating objects: CreateObject and CreateSessionObject. Session Objects are objects that are only active for the current session and are not persisted in the model. Let’s leverage a Session Object here; our objects will be created on the fly every time we load up our dashboard.

Because we are going to build multiple objects from our App instance, we can slightly refactor the code so that we have a re-usable Promise for our App instance stored in a variable:

With a working Generic Object in place, let’s go back to our object definition and add a dynamic property for the ListObjectDef. The full specification for a ListObjectDef can be found in the API docs. It has a lot of options for controlling how the list object works. For this list object, we want to use the field “Region”. We are going to keep things simple and just configure it to show that field, with all of there other options remaining undefined and thus using the default values. Following the specification in the API docs, we can define the list object for “Region” by using an inline dimension definition, which allows us to specify a field based on field name via the qDef/qFieldDefs property. This property takes an array of field names, which can be cycled through. We will just define 1 field: “Region”:

With this definition, our Generic Object will now contain a list object for the field “Region”.

Setting up Data Fetching

There is 1 more property we need to set for our list object to be useful. In Chapter 102, we discussed how a Generic Objects properties are evaluated with the Engine to produce a layout. The layout includes data from the Associate Model. By default, when we get a layout for a Generic Object with a list object, the list object will actually return no field value data. This is to give us better control over managing what data should be returned, allowing us to efficiently work with larger data sets. To setup our Generic Objects so that the layouts return us the field value data, we need to use the qInitialDataFetch property of the ListObjectDef.

In the API documentation, the qInitialDataFetch property has the description “Fetches an initial data set”. This initial data set is what gets returned every time we make a GetLayout call. The qInitialDataFetch property takes in definitions for pages of data that will be returned. A data page is simply a table of data defined with a starting point and a size. For example, let’s say I have a 3 column table with 10 records in it, and I want to get columns 2 and 3 for rows 5 through 10.

I can define that segment of the table using the data page structure defined here. The definition structure is simple:

  • qLeft: the column position to start at. 0 corresponds with the first column

  • qWidth: the number of columns to pull from the starting position

  • qTop: the row position to start at. 0 corresponds with the first row

  • qHeight: the number of rows to pull from the starting position

For my example table, the data page to get the highlighted area would be defined by the JSON to the right.

When setting a data page size, there is a constraint on the amount of data that QIX will send that we must observe. In any page, the Engine will only send at most 10,000 cells of data. A cell is a combination of a row and column. In the page defined here, we have 2 columns and 6 rows. This would yield us 12 cells of data, well below the 10,000 cell limit. If you need to get more than 10,000 cells, you will have to manually request data pages with the GetListObjectData method documented here.

Let’s apply this concept to our Region listbox. We want to get all of the values in our Region field. Therefore, our starting row position will be 0, which is the first row. We know there are only 4 values in our field, so we can just set the height to 4. We only have 1 column, so we can just pull the first column. This translates into this data page:

Let’s add the qInitialDataFetch property to our Generic Object definition file:

Now when we evaluate layouts on our Generic Object, we will get a data set of all 4 field values back.

Getting the ListObject Layouts

Our goal is to render the field values of the Region field on the front end of our dashboard. In order to get these values, we need to evaluate our properties through the Engine to produce a layout. The layout can be retrieved with the GetLayout method:

However, this code will only execute our layout call one time. Recall that our List Object is a dynamic structure. As we filter the data model, the data in our List Object will change. For example, the field values will have different selection states, depending on what has been filtered. We will want to render our filter based on the selection states of the values, so we need to keep our layouts up to date with the data model.

In Chapter 104, we talked about the invalidation process, where Generic Objects change on the server and are no longer in sync with the latest layout we’ve pulled for them. Enigma notifies us of invalidation events for an instance using a “changed” event. We can hook into this event and provide a callback that will execute every time the object invalidates. We can validate our object within that callback by making another GetLayout call:

Now we will retrieve a new layout anytime the object changes.

Rendering a Filter with the Layout

If you inspect the properties of your layout in the console, you will see that instead of qListObjectDef, we get a property called qListObject. The various properties of this calculated layout are covered in depth in the API documentation. For our purposes here, let’s look at the property that has the data we need to render:qDataPages.

qDataPages, located in our layout via layout.qListObject.qDataPages, contains an Array of data pages. Recall that with our qInitialDataFetch property, we defined a single data page that would capture the first 4 rows and first column of our field data. We can now access this data page, stored in the first and only member of our qDataPages array.

We can access this page using layout.qListObject.qDataPages[0]. This will return for us the page, which will have several properties. Our calculated data resides in a property called qMatrix which is an array of arrays that form a table structure.

Each element in the qMatrix array corresponds with 1 row of data. If we wanted to get the 2nd row of the table, we would access it like so: qMatrix[1]

Each cell is a JSON object. That object has several properties to describe the value in that cell. These properties could include:

  • qText – a text representation of the cell value

  • qNum – a numeric representation of the cell value

  • qElemNumber – a rank number of the cell value. For dimensions, this corresponds with the FieldValueIndex of the dimension value, which we will use later for selection

  • qState – the selection state of the field value

There are other properties that you may find in a cell; to learn more, see NxCell ‒ Qlik Sense.

Using this structure, we can easily pull out values from the calculated data. In the example above, we would get the numeric value of the cell in the 2nd row, third column of the qMatrix viaqMatrix[2][1].qNum, which would return the number "1".

Let’s do an example with our List Object. Say we want to get the text value of the first field value. We could do it like so:

  • First we access the layout.

  • Then we access the List Object.

  • Next we open the first element of qDataPages, because we only have 1 page defined.

  • On that page, we access the qMatrix and select the 1st row (index 0).layout.qListObject.qDataPages[0].qMatrix[0]

  • In that row, we grab the 1st column (index 0).layout.qListObject.qDataPages[0].qMatrix[0][0]

  • Finally, we grab the text representation of the value.layout.qListObject.qDataPages[0].qMatrix[0][0].qText

Let’s take these values and print them to our page in an unordered list. First we’ll need to modify our index.html file to include elements that we can render to. Update the <body> with the following contents:

We’ve added a column for the filters and a column for our chart, as well unordered lists for the filters and a div for the chart. We will position these properly with CSS later.

For now, let’s write a function that will take in an HTML element and our layout and render the field values as list items to the element. We’ll add this code in a modular fashion again, rather than cramming everything into index.js. Create a new file in the src folder called render-filter.js. This file will export our function using CommonJS syntax:

Let’s update our index.js file to load in this function and then leverage it whenever we get a layout for the Region list object. To import the function, add a require() statement to the top of the file:

Then, we can use the function further down with our Region generic object layouts:

We get the #region ul that we created in index.html and pass it to the renderFilter function along with our layouts. We can see the result in our dashboard, but it won't look exactly inspiring. Let’s add some CSS to clean it up.

In our stylesheet, we changed the font, sized our filter-column, and modified the typography and spacing of the filter components. This styling includes specific styles for field values that are selected or excluded.

Making Selections with the Filter

Our filter looks nice, but it’s useless if we can’t interact with it to actual modify the state of our data model. Let’s wire it up to make selections. Our desired behavior is that when we click on a filter value, it should select that value. If the value has already been selected, it should remove the selection.

To make List Object selection calls, we can use a Generic Object method called SelectListObjectValues. This method takes in a path string that describes where our List Object is defined in our Generic Object, as well as the element numbers that we want to select and a toggle mode. Let’s go through those one by one.

The path string is like an address for our List Object definition. Our Generic Object can actually hold multiple dynamic property definitions, including multiple List Objects. When making a selection, we need to tell the Generic Object which structure we are making a selection on. The path syntax uses forward slashes to denote sub-properties. For our current Generic Object, this translates to a path of “/qListObjectDef”.

The element numbers are passed in as an Array of Integers. Earlier, we looked at the qMatrix structure that contains our calculated data. Specifically, we looked at the individual cells in our row and column structure and examined properties like qNum and qText. One special property we discussed was the qElemNumber. When looking at a dimension cell, the qElemNumber represents the Field Index of a dimension value within a dimension. The Field Index is QIX’s unique identifier for a distinct value in a Field. This index is important to us because it can be used to tell the Engine exactly which values we want to filter in a dimension.

We will pass the field index, aka the element number, to our selection API call on click.

Toggle mode determines whether a selection is added to the already existing set of selections, or if it completely overrides the previous selections. A boolean is provided to enable or disable it. When toggle mode is true, the selections are cumulative. I.e., if I select element 3, and then select element 4, both 3 and 4 will be selected. If toggle mode is false, previous selection states are discarded. In that scenario, selecting element 4 after selecting element 3 will discard that previous selection, leaving only element 4 selected. For our filter, we will keep toggle mode set to true.

Let’s say I wanted to select the very first field value in the Region filter. The API call would be written as:regionLB.selectListObjectValues(“/qListObjectDef”, [0], true)`

For our filter, we want to dynamically execute these calls based on what list item is clicked. Therefore, we need to update our rendering function.

In order to make API calls, our rendering function needs access to the Generic Object for our filter. We will add it as an input parameter for the function. Then, we can add a click event listener for each list item that will execute the SelectListObjectValues call when a list item is clicked. The click event will pass the corresponding element number to the call.

Now we just need to update our render calls in index.js to pass the Generic Object to the function, and our listbox will be selectable:

We now have a working filter on our web app.

We can easily add a second filter for State now by creating a Generic Object for it and then reusing our renderFilter function. First, let’s create a state-listobject.json file in our defs folder:

Then, we can load that in our index.js file and build a filter with it:

The complete index.js file should look like so:

We should now have two working list boxes on our page.