Some help please

[wasab]

I am having a huge headache trying to increase performance of a post request. This is the route where I am having a performance issue.

router.post('/search', async function (req, res) {
    try{
        let timestamp = (req.body.timestamp) ? req.body.timestamp * 1000 : Date.now();
        let limit = (req.body.limit) ? req.body.limit : 25;
        if(limit > 100) throw new Error('limit can not be more than 100');

        req.body.replies = (typeof req.body.replies === 'undefined') ? true : req.body.replies;
        // first filters by time stamp
        let items = await Item.find({
            createdAt: {$lte: new Date(timestamp).toISOString()} // find the items in which the timestamp is less or equal
        });
        let responseItems = [];
        for(let i = 0; i < items.length; i++){
            let current_item = items[i];
            let current_item_user = await User.findOne({_id: current_item._userId});
            if(!current_item_user) throw new Error('current item user not found');

            // filter by user name if username is provided
            if(req.body.username){
                if(req.body.username !== current_item_user.username) continue;
            };

            // filtered by items if followed by the loggined user
            if(req.body.following){
                if(!req.isAuthenticated()) {
                    throw new Error('user needs to be logged in to use following in advance search');
                }
                if(!req.user.following.includes(current_item_user.username)) continue;
            }

            // if query string is pass
            if(req.body.q && (req.body.q !== '')){
                let wordlist = req.body.q.toLowerCase().split(' ');
                let content = current_item.content.toLowerCase().split(' ');
                if(!(wordlist.some(r=> content.indexOf(r) >= 0))) continue;
            }

            // if hasMedia is passed
            if(req.body.hasMedia){
                if(current_item.media.length === 0) continue;
            }

            if(req.body.parent){
                if(!(current_item._parentId === req.body.parent)) continue;
            }

            if(!req.body.replies){
                if(current_item.childType === "reply") continue;
            }

            let current_json = {
                id: current_item._id,
                username: current_item_user.username,
                property: {likes: current_item.likes},
                retweeted: current_item.retweeted,
                content: current_item.content,
                timestamp: new Date(current_item.createdAt).getTime(),
                media: current_item.media,
                parent: current_item._parentId,
                media: current_item.media,
                parent: (req.body.replies) ? current_item._parentId : undefined,
                childType: current_item.childType
            }

            responseItems.push(current_json);
        }


        // return the limit number of items
        responseItems = responseItems.slice(0, limit);
        return res.send({status: "OK", items: responseItems});
    }catch (err) {
        return res.status(500).send({status: "error", error: err.message});
    }
});

The response time is a bloody 1600ms and sometimes all the way to 2000ms which is already unacceptable.

 

For my final project grading, my professor will use a script that will literally spam my server with thousands of clients and requests all at once. If it already does this poorly when debugging, I do not see how it can survive the torture. 

 

Can anyone offer me some tips to speed it up? what should I do? rewrite the entire loop? Add more query parameters when querying the database? use memcached? Or something else?  I also have more machines at my disposal to scale out if it's hardware limited but I doubt it is the case here. Can an experienced devs help me out?

[Sauron]

I don't see anything that immediately stands out as a performance bottleneck, try removing everything and just returning 200 - if that also takes so long then the problem is likely with your connection and not with the code.

[wasab]

4 minutes ago, Sauron said:

I don't see anything that immediately stands out as a performance bottleneck, try removing everything and just returning 200 - if that also takes so long then the problem is likely with your connection and not with the code.

If I remove that loop, response time drops to 300ish ms. Bottleneck seems to be there. How should I improve this? Nodejs runs on single thread so not like I can do some clever multi threading here. The only thing I can think of is to pass some of the filtering to the database query instead of doing it in node.js but I am having a bad time in coming up with a way to do this. 

[Sauron]

13 minutes ago, wasab said:

let current_item_user = await User.findOne({_id: current_item._userId});

Where is this data coming from? If this call is slow and it's run multiple times in the loop it could explain the high latency.

 

Other than that you can try removing things from the loop and seeing which ones make a difference, then you can try to figure out why.

[wasab]

9 minutes ago, Sauron said:

Where is this data coming from? If this call is slow and it's run multiple times in the loop it could explain the high latency.

 

Other than that you can try removing things from the loop and seeing which ones make a difference, then you can try to figure out why.

Coming from the mongo database. Item schema has a field _userId which I am using to reference the correct documents in another schema, the user. Is there any optimization I can do here? 

Edited November 15, 2019 by wasab
Mongo*

[Sauron]

7 minutes ago, wasab said:

Coming from the mongo database. Item schema has a field _userId which I am using to reference the correct documents in another schema, the user. Is there any optimization I can do here? 

You could get all the data you need in one query instead of accessing the DB multiple times. With multiple requests, any latency from mongo is going to stack. If you make 100 querys and each query has a latency of 10ms, which is reasonable, you end up with a full second of latency where 10ms would have sufficed.

[mariushm]

Yeah, make a user.findMultiple  function  and send that function your IDs as an array or a string separated by some character and get back an array of records

 

[DevBlox]

300ms seems to a be a bit much for an empty request . Another thing you can try, is using pm2 and spawning more processes, to handle more requests, since JS is single threaded, you might have a bit of an edge.

[wasab]

8 hours ago, DevBlox said:

300ms seems to a be a bit much for an empty request . Another thing you can try, is using pm2 and spawning more processes, to handle more requests, since JS is single threaded, you might have a bit of an edge.

my professor said 700ms is the threshold so i am happy with 700ms and less. 

 

If one single process handles it at 2000ms, it wont help by pawning more process though. My web page still waits for a second and two before it fully loads. What I am trying to do is to get the latency down to 700ms or less then I will set up a load balancer and spawn more proccess to handle more requests. I am not sure if i am reasoning this right.... 

[Eigenvektor]

9 hours ago, wasab said:

Coming from the mongo database. Item schema has a field _userId which I am using to reference the correct documents in another schema, the user. Is there any optimization I can do here? 

Does your database have indexes? Creating an index that covers your query parameters and/or your foreign keys can speed things up a lot, depending on the amount of data you have.

 

https://docs.mongodb.com/manual/applications/indexes/

[Franck]

2 hours ago, wasab said:

will set up a load balancer and spawn more proccess to handle more requests

You can instantiate process now in JS ? my JS is a bit old but in my days we could instantiate threads not process.

 

But yes spawning more threads help to a certain extend. If your data size is reasonable and it's not what is limiting you then a couple threads will help you.

[wasab]

1 hour ago, Franck said:

You can instantiate process now in JS ? my JS is a bit old but in my days we could instantiate threads not process.

 

But yes spawning more threads help to a certain extend. If your data size is reasonable and it's not what is limiting you then a couple threads will help you.

I don't think JS ever supports threads to begin with. It's event driven and asynchronous but that is its main strength and one reason I picked it over python flask. I won't have to wait for a database to spit back a result before I can begin processing the next request like in python flask. 

 

By spawning proccesses, I mean just keep on launching more of them in terminal, maybe on different machine as well and use round robin proxy in nginx to load balance. 

 

I am gonna try some of the suggestions above and see if get anywhere. 

[mariushm]

No, can't really do threads. You can have async stuff... you could use Promises to do multiple requests in parallel and process them the results as the results arrive back, but it's not really multithreaded.

This sounds like something where Go with its built in concurrency stuff would shine.

 

[camjocotem]

let current_item_user = await User.findOne({_id: current_item._userId});

Couldn't you restructure your code so instead of awaiting on each iteration you create an array outside the loop, populate the array with promises and then await with Promise.all and iterate over all the responses?

 

Edit:  Just saw your post mariushm, just realised I said the same thing as you lol

[wasab]

1 hour ago, camjocotem said:

let current_item_user = await User.findOne({_id: current_item._userId});

Couldn't you restructure your code so instead of awaiting on each iteration you create an array outside the loop, populate the array with promises and then await with Promise.all and iterate over all the responses?

 

Edit:  Just saw your post mariushm, just realised I said the same thing as you lol

That is still going to make requests to the mongodb for every item in the array. It makes no difference.

 

I just need to find a query that's equal to the join on foregin keys in SQL, that way I get everything in one go.

 

I am kinda procrastinating on improving performance at the moment and currently focusing on just finishing off rest of the unimplemented features but thanks for the suggestion. 

[camjocotem]

13 hours ago, wasab said:

That is still going to make requests to the mongodb for every item in the array. It makes no difference.

 

I just need to find a query that's equal to the join on foregin keys in SQL, that way I get everything in one go.

 

I am kinda procrastinating on improving performance at the moment and currently focusing on just finishing off rest of the unimplemented features but thanks for the suggestion. 

Have you tried it? 

Currently it's effectively running synchronously where it must wait for the given iteration's query to finish before, it looks like you could fire off a lot of requests at the same time and then wait at the end of that loop for the results.

 

Failing that you could use mongo's $in statement with something like

let userIds = items.map(i => i._userId);
let users = await User.find({ _id: { $in: userIds } });

 

[wasab]

2 hours ago, camjocotem said:

Have you tried it? 

Currently it's effectively running synchronously where it must wait for the given iteration's query to finish before, it looks like you could fire off a lot of requests at the same time and then wait at the end of that loop for the results.

 

Failing that you could use mongo's $in statement with something like

let userIds = items.map(i => i._userId);
let users = await User.find({ _id: { $in: userIds } });

 

I had, the speed is still abysmal 

 

This is my reworked loop 

// TODO finishes the rest of the filter
router.post('/search', async function (req, res) {
    // if(process.env.PRINT_REQUESTS === 'true') console.log(req.body);
    try{
        let timestamp = (req.body.timestamp) ? req.body.timestamp * 1000 : Date.now();
        let limit = (req.body.limit) ? req.body.limit : 25;
        let ranking = (req.body.rank) ? req.body.rank : "interest";
        if(limit > 100) throw new Error('limit can not be more than 100');

        req.body.replies = (typeof req.body.replies === 'undefined') ? true : req.body.replies;
        // first filters by time stamp
        let items = await Item.find({
            createdAt: {$lte: new Date(timestamp).toISOString()} 
        });


       let responseItems = await Promise.all(items.map(
           async (current_item) => {
                let current_item_user = await User.findOne({_id: current_item._userId});
                if(!current_item_user) throw new Error('current item user not found');

                // filter by user name if username is provided
                if(req.body.username){
                    if(req.body.username !== current_item_user.username) return;
                };

                // filtered by items made by the user the logged in user is following
                if(req.body.following){
                    if(!req.isAuthenticated()) {
                        throw new Error('user needs to be logged in to use following in advance search');
                    }
                    if(!req.user.following.includes(current_item_user.username)) return;
                }

                // if query string is pass
                if(req.body.q && (req.body.q !== '')){
                    let wordlist = req.body.q.toLowerCase().split(' ');
                    let content = current_item.content.toLowerCase().split(' ');
                    if(!(wordlist.some(r=> content.indexOf(r) >= 0))) return;
                }

                // if hasMedia is passed
                if(req.body.hasMedia){
                    if(current_item.media.length === 0) return;
                }

                if(req.body.parent){
                    if(!current_item._parentId) return;
                    if(!(current_item._parentId.toString() === req.body.parent)) return;
                }

                if(!req.body.replies){
                    if(current_item.childType === "reply") return;
                }

                let current_json = {
                    id: current_item._id,
                    username: current_item_user.username,
                    property: {likes: current_item.likes},
                    retweeted: current_item.retweeted,
                    content: current_item.content,
                    timestamp: new Date(current_item.createdAt).getTime(),
                    media: current_item.media,
                    parent: current_item._parentId,
                    media: current_item.media,
                    parent: (req.body.replies) ? current_item._parentId : undefined,
                    childType: current_item.childType
                }
                return current_json;
            }
        ));

        if(ranking === "time"){
            responseItems.sort((current, next) => {{
                return (current.timestamp > next.timestamp) ? -1 : 1; 
        } else if(ranking === "interest"){
            responseItems.sort((current, next) => {{
                return ((current.property.likes + current.retweeted) > (next.property.likes + next.retweeted)) ? -1 : 1;
            }});
        }

        // return the limit number of items
        responseItems = responseItems.slice(0, limit);
        return res.send({status: "OK", items: responseItems});
    }catch (err) {
        return res.status(500).send({status: "error", error: err.message});
    }
});

 

Did I do something wrong here? Or is it that the system is just slow? I am currently using private cloud servers provided by my school and the processors seem to be Intel duo core2 

[wasab]

2 hours ago, camjocotem said:

 

let userIds = items.map(i => i._userId);
let users = await User.find({ _id: { $in: userIds } });

 

Can this be combine with Item.find into one single document like a sql join? That way call to mongo is literally reduce to just one. 

[Eigenvektor]

24 minutes ago, wasab said:

Can this be combine with Item.find into one single document like a sql join? That way call to mongo is literally reduce to just one. 

One or two queries shouldn't make too much of a difference. The important part would be to get rid of the repeat queries inside the loop.

But I suppose this is what you'd need: https://www.w3schools.com/nodejs/nodejs_mongodb_join.asp

 

Have you tried logging timing information? This way you could get a better understanding which parts of your code are taking the most time and are most in need of optimization.

[camjocotem]

51 minutes ago, wasab said:

Did I do something wrong here? Or is it that the system is just slow? I am currently using private cloud servers provided by my school and the processors seem to be Intel duo core2 

I was thinking of something a bit more like this:

        let promiseList = []

        items.forEach((item) => {
            promiseList.push(User.findOne({_id: item._userId}));
        })

        let userList = await Promise.all(promiseList);

        for (let i = 0; i < items.length; i++) { 
            let current_item = items[i];
            let current_item_user = userList.find(user => user._id === current_item._userId);

            /// Rest of code here
        }

 

[wasab]

16 hours ago, camjocotem said:

I was thinking of something a bit more like this:

        let promiseList = []

        items.forEach((item) => {
            promiseList.push(User.findOne({_id: item._userId}));
        })

        let userList = await Promise.all(promiseList);

        for (let i = 0; i < items.length; i++) { 
            let current_item = items[i];
            let current_item_user = userList.find(user => user._id === current_item._userId);

            /// Rest of code here
        }

 

 

I am not sure what's going on here but latency is now at 24959 ms

My current code:

// TODO finishes the rest of the filter
router.post('/search', async function (req, res) {
    // if(process.env.PRINT_REQUESTS === 'true') console.log(req.body);
    try{
        let timestamp = (req.body.timestamp) ? req.body.timestamp * 1000 : Date.now();
        let limit = (req.body.limit) ? req.body.limit : 25;
        let ranking = (req.body.rank) ? req.body.rank : "interest";
        if(limit > 100) throw new Error('limit can not be more than 100');

        req.body.replies = (typeof req.body.replies === 'undefined') ? true : req.body.replies;
        // first filters by time stamp
        let items = await Item.find({
            createdAt: {$lte: new Date(timestamp).toISOString()} // find the items in which the timestamp is less or equal
        });

        let promiseList = []

        items.forEach((item) => {
            promiseList.push(User.findOne({_id: item._userId}));
        })

        let userList = await Promise.all(promiseList);

        let responseItems = [];
        for (let i = 0; i < items.length; i++) {
            let current_item = items[i];
            let current_item_user = userList.find(user => user._id.equals(current_item._userId));
            if(!current_item_user) throw new Error('current item user not found');

            // filter by user name if username is provided
            if(req.body.username){
                if(req.body.username !== current_item_user.username) return;
            };

            // filtered by items made by the user the logged in user is following
            if(req.body.following){
                if(!req.isAuthenticated()) {
                    throw new Error('user needs to be logged in to use following in advance search');
                }
                if(!req.user.following.includes(current_item_user.username)) return;
            }

            // if query string is pass
            if(req.body.q && (req.body.q !== '')){
                let wordlist = req.body.q.toLowerCase().split(' ');
                let content = current_item.content.toLowerCase().split(' ');
                if(!(wordlist.some(r=> content.indexOf(r) >= 0))) return;
            }

            // if hasMedia is passed
            if(req.body.hasMedia){
                if(current_item.media.length === 0) return;
            }

            if(req.body.parent){
                if(!current_item._parentId) return;
                if(!(current_item._parentId.toString() === req.body.parent)) return;
            }

            if(!req.body.replies){
                if(current_item.childType === "reply") return;
            }

            let current_json = {
                id: current_item._id,
                username: current_item_user.username,
                property: {likes: current_item.likes},
                retweeted: current_item.retweeted,
                content: current_item.content,
                timestamp: new Date(current_item.createdAt).getTime(),
                media: current_item.media,
                parent: current_item._parentId,
                media: current_item.media,
                parent: (req.body.replies) ? current_item._parentId : undefined,
                childType: current_item.childType
            }
            responseItems.push(current_json);
        }




        if(ranking === "time"){
            responseItems.sort((current, next) => {{
                return (current.timestamp > next.timestamp) ? -1 : 1; // if current is greater than next then return -1 else 1. -1 comes first
            }});
        } else if(ranking === "interest"){
            responseItems.sort((current, next) => {{
                return ((current.property.likes + current.retweeted) > (next.property.likes + next.retweeted)) ? -1 : 1;
            }});
        }

        // return the limit number of items
        responseItems = responseItems.slice(0, limit);
        return res.send({status: "OK", items: responseItems});
    }catch (err) {
        return res.status(500).send({status: "error", error: err.message});
    }
});

 

Edited November 20, 2019 by wasab
Returns should be continues.

[Eigenvektor]

1 hour ago, wasab said:

I am not sure what's going on here but latency is now at 24959 ms

You still have multiple queries this way and I/O is expensive. Plus, you now have the overhead of searching through your list every iteration with:

userList.find(user => user._id.equals(current_item._userId));

which is most likely even slower than:

await User.findOne({_id: current_item._userId});

So you've essentially traded one query each iteration with all the queries upfront and one lookup each iteration.

 

Doing multiple queries upfront doesn't net you much, if you have to wait until they're all done, because you're essentially back to serial operation. Plus, as I said, you're now iterating over your list of users each iteration to get the user from your list instead of the database.

 

I would definitely try the single-query variant and, if possible, combine it with the sort operation directly.

[Eigenvektor]

This should do it, right? Sorry if there's a syntax error, I can't really test the code

Spoiler

// TODO finishes the rest of the filter
router.post('/search', async function (req, res) {
    // if(process.env.PRINT_REQUESTS === 'true') console.log(req.body);
    try {
        let timestamp = (req.body.timestamp) ? req.body.timestamp * 1000 : Date.now();
        let limit = (req.body.limit) ? req.body.limit : 25;
        let ranking = (req.body.rank) ? req.body.rank : "interest";
        if (limit > 100) throw new Error('limit can not be more than 100');

        req.body.replies = (typeof req.body.replies === 'undefined') ? true : req.body.replies;
        // first filters by time stamp
        let items = await Item.find({
            createdAt: { $lte: new Date(timestamp).toISOString() } // find the items in which the timestamp is less or equal
        });

        let userIds = items.map(i => i._userId);
        let users = await User.find({ _id: { $in: userIds } });

        // Should probably check for users != empty here and throw otherwise

        let responseItems = [];
        for (let i = 0; i < users.length; i++) {
            let current_item_user = users[i];
            if (!current_item_user) throw new Error('current item user not found');

            // filter by user name if username is provided
            if (req.body.username) {
                if (req.body.username !== current_item_user.username) return;
            };

            // filtered by items made by the user the logged in user is following
            if (req.body.following) {
                if (!req.isAuthenticated()) {
                    throw new Error('user needs to be logged in to use following in advance search');
                }
                if (!req.user.following.includes(current_item_user.username)) return;
            }

            // if query string is pass
            if (req.body.q && (req.body.q !== '')) {
                let wordlist = req.body.q.toLowerCase().split(' ');
                let content = current_item.content.toLowerCase().split(' ');
                if (!(wordlist.some(r => content.indexOf(r) >= 0))) return;
            }

            // if hasMedia is passed
            if (req.body.hasMedia) {
                if (current_item.media.length === 0) return;
            }

            if (req.body.parent) {
                if (!current_item._parentId) return;
                if (!(current_item._parentId.toString() === req.body.parent)) return;
            }

            if (!req.body.replies) {
                if (current_item.childType === "reply") return;
            }

            let current_json = {
                id: current_item._id,
                username: current_item_user.username,
                property: { likes: current_item.likes },
                retweeted: current_item.retweeted,
                content: current_item.content,
                timestamp: new Date(current_item.createdAt).getTime(),
                media: current_item.media,
                parent: current_item._parentId,
                media: current_item.media,
                parent: (req.body.replies) ? current_item._parentId : undefined,
                childType: current_item.childType
            }
            responseItems.push(current_json);
        }

        if (ranking === "time") {
            responseItems.sort((current, next) => {
                {
                    return (current.timestamp > next.timestamp) ? -1 : 1; // if current is greater than next then return -1 else 1. -1 comes first
                }
            });
        } else if (ranking === "interest") {
            responseItems.sort((current, next) => {
                {
                    return ((current.property.likes + current.retweeted) > (next.property.likes + next.retweeted)) ? -1 : 1;
                }
            });
        }

        // return the limit number of items
        responseItems = responseItems.slice(0, limit);
        return res.send({ status: "OK", items: responseItems });
    } catch (err) {
        return res.status(500).send({ status: "error", error: err.message });
    }
});

 

 

[wasab]

1 hour ago, Eigenvektor said:

This should do it, right? Sorry if there's a syntax error, I can't really test the code

  Hide contents

// TODO finishes the rest of the filter
router.post('/search', async function (req, res) {
    // if(process.env.PRINT_REQUESTS === 'true') console.log(req.body);
    try {
        let timestamp = (req.body.timestamp) ? req.body.timestamp * 1000 : Date.now();
        let limit = (req.body.limit) ? req.body.limit : 25;
        let ranking = (req.body.rank) ? req.body.rank : "interest";
        if (limit > 100) throw new Error('limit can not be more than 100');

        req.body.replies = (typeof req.body.replies === 'undefined') ? true : req.body.replies;
        // first filters by time stamp
        let items = await Item.find({
            createdAt: { $lte: new Date(timestamp).toISOString() } // find the items in which the timestamp is less or equal
        });

        let userIds = items.map(i => i._userId);
        let users = await User.find({ _id: { $in: userIds } });

        // Should probably check for users != empty here and throw otherwise

        let responseItems = [];
        for (let i = 0; i < users.length; i++) {
            let current_item_user = users[i];
            if (!current_item_user) throw new Error('current item user not found');

            // filter by user name if username is provided
            if (req.body.username) {
                if (req.body.username !== current_item_user.username) return;
            };

            // filtered by items made by the user the logged in user is following
            if (req.body.following) {
                if (!req.isAuthenticated()) {
                    throw new Error('user needs to be logged in to use following in advance search');
                }
                if (!req.user.following.includes(current_item_user.username)) return;
            }

            // if query string is pass
            if (req.body.q && (req.body.q !== '')) {
                let wordlist = req.body.q.toLowerCase().split(' ');
                let content = current_item.content.toLowerCase().split(' ');
                if (!(wordlist.some(r => content.indexOf(r) >= 0))) return;
            }

            // if hasMedia is passed
            if (req.body.hasMedia) {
                if (current_item.media.length === 0) return;
            }

            if (req.body.parent) {
                if (!current_item._parentId) return;
                if (!(current_item._parentId.toString() === req.body.parent)) return;
            }

            if (!req.body.replies) {
                if (current_item.childType === "reply") return;
            }

            let current_json = {
                id: current_item._id,
                username: current_item_user.username,
                property: { likes: current_item.likes },
                retweeted: current_item.retweeted,
                content: current_item.content,
                timestamp: new Date(current_item.createdAt).getTime(),
                media: current_item.media,
                parent: current_item._parentId,
                media: current_item.media,
                parent: (req.body.replies) ? current_item._parentId : undefined,
                childType: current_item.childType
            }
            responseItems.push(current_json);
        }

        if (ranking === "time") {
            responseItems.sort((current, next) => {
                {
                    return (current.timestamp > next.timestamp) ? -1 : 1; // if current is greater than next then return -1 else 1. -1 comes first
                }
            });
        } else if (ranking === "interest") {
            responseItems.sort((current, next) => {
                {
                    return ((current.property.likes + current.retweeted) > (next.property.likes + next.retweeted)) ? -1 : 1;
                }
            });
        }

        // return the limit number of items
        responseItems = responseItems.slice(0, limit);
        return res.send({ status: "OK", items: responseItems });
    } catch (err) {
        return res.status(500).send({ status: "error", error: err.message });
    }
});

 

 

I do not see how you can determine the correct item user by just indices however since the relationship is not one to one(one user can have many items). The route is suppose to return an array of items along with their owner's usernames btw. 

[Eigenvektor]

1 minute ago, wasab said:

I do not see how you can determine the correct item user by just indices however since the relationship is not one to one(one user can have many items).

Oh, I see, you need items not users, and for each item you need the associated user? Then I suppose a Join would be your best option.

 

In any case, your problem right now is that you replaced the MongoDB "find" with a JavaScript "find", which is very slow: https://nikitahl.com/how-to-find-an-item-in-a-javascript-array/. You haven't reduced the number of queries and added more overhead by repeatedly searching through the results.