Swift - Filtering data over multiple collections

Hello everyone,

I am stuck with one issue. I am using the InstantSearch api to display results in a collection view.
I have three entities: Challenge, Participation and Rating.

Whenever I search for challenges, I can easily display them in my collection view since this is the “main” object. On the entities for rating and participation, I just have a reference to a challenge (challengeId attribute).

So now, when I search in the rating index as an example, I get the rating objects. How I can use filter now the rating objects according to the title attribute which is only available in the challenge index?

What would be the best solution here?
Many thanks in advance and stay healthy!

Best regards,
Kaan

Hi @kaan548,

If I understand correctly you have 3 entities which are related and you aren’t sure how to structure your data to get the best search experience.

This documentation on handling data relationships may be helpful.

Hello @cindy.cullen

Thanks for the reply. I actually know how to structure the data. As in the example documentation, the entities are connected with an attribute.

But is also says that is a JSON join request. How can I do that using the InstantSearch?

Best regards,
Kaan

Maybe just to give a simple example, how my data is structured:

Challenge

  • Id
  • Title
  • Description

Participation

  • Id
  • ChallengeId
  • UserId

What I need now (trying to explain with SQL) is following:

Select Challenge.Title, Challenge.Description From Participation ON Participation.ChallengeId = Challenge.Id Where Participation.UserId = "Foo"

How can I achieve that?

Hi @kaan548, you can not join indices like you can SQL database tables, so you will need to aggregate the data into one index, or handle the connection between the two indices in your own code. Think of the search as more like a Google search as opposed to a database search. If you aggregate the data before you index and store the aggregated data in your indices, you should be able to search using filters to get the desired results.

{
   "challenge_id": 1,
   "challenge_title": "abc",
   "challenge_description": "....",
   "participation_id: 1,
   "user_id": '1',
  ...
}

Then you can get the challenge titles when you search by userId.

hmm… storing all the data into one index is actually not my intension.
What I achieved so far is following:

I fetch all the participation objects and for each participation object, I can fetch the related challenge object.

func collectionView(_ collectionView: UICollectionView, cellForItemAt indexPath: IndexPath) -> UICollectionViewCell {
        let myChallengesCell = collectionView.dequeueReusableCell(withReuseIdentifier: "myChallenges", for: indexPath) as! MyChallengesCollectionViewCell
            
  ....
    
    else if (challengeStateSegmentedControl.selectedSegmentIndex == 1) {
        let participationObject = Participation(json: (hitsCollectionViewController.hitsSource?.hit(atIndex: indexPath.row))!)
        
        self.algoliaSearch.fetchAllChallengesByIdAndSearchStringAndState(completionHandler: { (challengeObject) in
            if (challengeObject.title.isEmpty) {
                return
            }
        
            myChallengesCell.challengeTitleLabel.text = challengeObject.title
            myChallengesCell.fetchChallengeImageById(challengeObject: challengeObject)
            myChallengesCell.setRemainingTime(challengeObject: challengeObject)
            myChallengesCell.fetchRatingForChallengeContent(userObject:self.loggedInUserObject , challengeObject: challengeObject)
        }, challengeId: participationObject.challengeId, searchString: searchBarController.searchBar.text!, challengeState: SELECTED_CHALLENGE_STATE)
    }

The question is, what happens if I search for a challenge by a filter value and there is no result? I would have an empty cell in my collection view or?
What would be nice is fetching all the challenges as I am doing above for each participation objects and then doing the filtering locally. This would mean, I basically modify the datasource of the hitsCollectionViewController which is set to read-only though.

Any other ideas/approaches?

There is additionally one thing which I also don’t understand. I have a property class where I store the indices with the ids such as:

let challengeIndexSearcher = SingleIndexSearcher(appID: "XXX", apiKey: "XXX", indexName: "challenges")
**let** challengeIndex: Index = Client(appID: "XXX", apiKey: "XXX").index(withName: "challenges")

Then in my view controller, where I fetch challenges, I have following code:

private func setupChallengeIndexSearchSettings() {
         let query = Query()
        query.filters = "userId:\(Auth.auth().currentUser?.uid) AND (title:\ 
          (searchBarController.searchBar.text) OR description:\(searchBarController.searchBar.text))"
        
        singleIndexHitsInteractor.connectController(hitsCollectionViewController)
        singleIndexHitsInteractor.connectSearcher(challengeIndexSearcher)
        
        challengeIndexSearcher.indexQueryState = .init(index: challengeIndex)
        singleIndexHitsInteractor.connectSearcher(challengeIndexSearcher)
        challengeIndexSearcher.search()
      }

Once this is executed, I can see that I get three results (as expected)

func collectionView(_ collectionView: UICollectionView, numberOfItemsInSection section: Int) -> Int {
        print(hitsCollectionViewController.hitsSource?.numberOfHits())
        return (hitsCollectionViewController.hitsSource?.numberOfHits())!
   }

Now the strange thing: I expect that cellForRow… is going trough the hitSource three times. It actually does, but after the three times, again

func collectionView(_ collectionView: UICollectionView, numberOfItemsInSection section: Int) -> Int

is called which again gives three objects as a result and then cellForRow is called. So instead of going through it 3 times, it goes though it 9 times.

Why is that the case?

I tried to change the content of the method to:

    private func setupChallengeIndexSearchSettings() {
        let xchallengeIndexSearcher = SingleIndexSearcher(appID: "XXX", apiKey: "XXX", indexName: "challenges")
        let xchallengeIndex: Index = Client(appID: "XXX", apiKey: "XXX").index(withName: "challenges")
        xchallengeIndexSearcher.indexQueryState = .init(index: xchallengeIndex)
        singleIndexHitsInteractor.connectSearcher(xchallengeIndexSearcher)
        xchallengeIndexSearcher.search()
    }

Above, I just wanted to avoid having a reference to the global index searcher variables since they are also used on other view controllers, but the approach above gives me no results at all.

Could it be that the filtering is not working properly?
I am executing:

 let query = Query()
    query.filters = "userId:4 SDhjkj (title:\(searchBarController.searchBar.text) OR description:\(searchBarController.searchBar.text))"
    
    singleIndexHitsInteractor.connectController(hitsCollectionViewController)
    singleIndexHitsInteractor.connectSearcher(challengeIndexSearcher)
    
    challengeIndexSearcher.indexQueryState = .init(index: challengeIndex, query: query)
    singleIndexHitsInteractor.connectSearcher(challengeIndexSearcher)
    challengeIndexSearcher.search()

This give me still 3 results where I would have expected 0 because of
"userId:4 SDhjkj

Hello @kaan548,

There are two approaches you can apply here.

As @cindy.cullen mentioned, the easiest way in the most proper approach is to merge your data into one index. Remember that the purpose of the search index is not to store or structure your data, but to provide the best and the most convenient search experience. The index is a mapping of your database for search purposes, but not the database itself. It can be, but it’s not designed for that.

Doing a manual join of the big indices on the client side can be really awkward and slow and risk to ruin the advantages of Algolia search engine from the user’s perspective.

So, the recommended solution is to drop the Participation index keeping just the Challenge and the User indices and to add the list of participants ids directly to your Challenge index:

Challenge

  • Id
  • Title
  • Description
  • Participants

The participants is the list of userIDs, so your usecase can be implemented directly by adding a filter for Participants field.

let searcher = SingleIndexSearcher(appID: “YourAppID”, apiKey: “YourApiKey”).index(withName: “Participants”) 
let userIDToSearch: String = “userIDtoSearch”
searcher.query.filters = “participants:\(userIDToSearch)”
searcher.search()

And that’s it!

If you absolutely need to keep all the data into separate tables, you can do as following:

  • Browse Participation index applying the filter by userID to get all the participations of required user (you have to use Index.browse method)
  • Extract the list of challengeID from Participation list you got on the previous step
  • Search in the Challenge index applying the as filter the list of challengeID: (“challengeId”:”id1” OR “challengeId”:”id2” OR “challengeId”:”idN”)

By doing this you get the results from challenge index for a concrete userID. Keep in mind that in this case you do N+1 search requests where N is the number of pages in search results for Participation table + one search in the Challenge index instead of just one search operation in the Participants index if you structure your data as I described above.

Regarding the InstantSearch related issues, could you precise which version of InstantSearch do you use?

Hi @vladislav.fitc

Many thanks for the reply. Regarding the version of my InstantSearch - it is 5.2.3. I saw that there is a newer version. I can update and check again.

Regarding your recommendation. You used this code:

let searcher = SingleIndexSearcher(appID: “YourAppID”, apiKey: “YourApiKey”).index(withName:
“Participants”)
let userIDToSearch: String = “userIDtoSearch”
searcher.query.filters = “participants:(userIDToSearch)”
searcher.search()

The index name must be Challenge instead of Participants or? I may have miscommunicated one thing. Just to understand the concept about fetching the data, I simplified the attributes for the participation index. Actually, I also have further fields such as

  • title
  • description
  • imageRef

So would you still recommend using the challenge index only? What about if I want to search for participation by using the title attribute?

And my Participation index has also a reference to another index called Rating.
One Participation object can have several Rating entries.

In some view controllers, I need to fetch all participation objects based on the challenge as well.

Best regards,
Kaan

Hi @kaan548,

I believe that was an oversight and the index name should be Challenge based on that example.

As Vlad mentioned, the purpose of the search index is not to store or structure your data, but to provide the best and the most convenient search experience. It is not a relational database and as such there is no easy or efficient way to achieve those types of tasks.

If each participant has several ratings, you could potentially condense to a list of participant+rating but that might get quite messy. I will defer to @vladislav.fitc to see if he has any further recommendations there.

@vladislav.fitc, do you have any recommendations?

Hi @kaan548,

Following your initial question:
Let’s assume you have a Challenge index containing the information about participants which looks as follows:

   [
    {
      "title": "First Challenge",
      "participants": ["p1", "p2”],
      "objectID": "Challenge1"
    },
    {
      "title": "Second Challenge",
      "participants": [ "p2", "p3”],
      "objectID": "Challenge2",
    },
    {
      "title": "Third Challenge",
      "participants": ["p3", "p4" ],
      "objectID": "Challenge3"
    }
 ]

Then you set participants as attribute for faceting in the settings of the index and then you can search for challenges by participants as follows:

let challengesIndex = SingleIndexSearcher(appID: “YourAppID”,  apiKey: “YourApiKey”, indexName: “Challenge”)
let userIDToSearch: String = “p1”
searcher.query.filters = “participants:\(userIDToSearch)”
searcher.search()

To search for multiple participants, the filters must look as follows:

searcher.query.filters = “participants:p1 OR participants:p2”

It’s really up to you how to structure your data, I am here to help you with the usage of InstantSearch and Algolia features.
The rule of thumb is that the index record must be an entity you consider to search. Answer yourself a question: “What do I search?” and add as many indices as the count of answers to this question. If you need more than one search operation to find a desired object, you are probably doing it wrong.

Hi @vladislav.fitc
I checked the InstantSearch related issues again after updating to version 7.1. They are still there. Could you have a look, please?

Also it could be worth to mention that

searcher.query.filters = “test”

can’t be used. Instead, you can use
searcher.query (string) = “test”

Thanks!

Hi @vladislav.fitc. Do we have any updates here? Thanks!

Hi @kaan548,

You can use

searcher.indexQueryState.query.filters = "..."

If you want to access the Query object of the searcher.

Do I get it well, that your issue is that you applied an incorrect string of filters and still got the results?

Did you consider to use FilterState connected to your Searcher to handle the filters in a safe manner?

Hello @vladislav.fitc,

yes, I applied a filter that was wrong but still got results. I haven’t used the FilterState . Thanks for the hint. I’ll try that as well.

Hi again @vladislav.fitc

   let xchallengeIndexSearcher = SingleIndexSearcher(appID: "XX", apiKey: "XX", indexName:"challenges")
    singleIndexHitsInteractor.connectController(self)
    singleIndexHitsInteractor.connectSearcher(xchallengeIndexSearcher)
       
    xchallengeIndexSearcher.search() 

The code above doesn’t give me any results. This can be seen in the console of Xcode:
2020-09-16T23:19:27+0200 info com.algolia.InstantSearchCore : InstantSearch.SingleIndexSearcher: received results - index: challenges query: "" hits count: 4 in 1ms

I mean, the index contains data and there is no filter. If I use the global indexSearcher variable that is basically used also in other view controllers, I get results but not always the correct ones. Can you checkt that please? Could it be that using one index searcher shares a session somehow?

When I look closely, I can actually see that I get a hits count of 4. That’s good but why is

        print(hitsSource?.numberOfHits())

returning 0?

Also, in the console, I can see this message:
NSErrorFailingURLStringKey=https://<appId>-dsn.algolia.net/1/indexes/*/queries
When I enter that url into the browser, it says the indexName would be invalid:
{"message":"indexName is not valid","status":400}

How can this be the case?

Hi @vladislav.fitc and @kevin.sullivan. Do we have any update here? Thanks!