Removing duplicates of a JSON dict in pythong

[Shally]

Hola,

So let's say I have this JSON:
 

{  
 "items": [
        {
            "itemID": "1",
            "itemInfo": [
                {
                    "name": "movieId",
                    "value": "1"
                },
                {
                    "name": "movieTitle",
                    "value": "Toy Story (1995)"
                }
            ]
        },
        {
            "itemID": "1",
            "itemInfo": [
                {
                    "name": "movieId",
                    "value": "2"
                },
                {
                    "name": "movieTitle",
                    "value": "Toy Story 2"
                }
            ]
        }
    ]
}

 

And I need to remove any items with the same itemIds, even if their itemInfo's are different.  Here's what I tried that didn't work:

items = local_request.extract_value(json_body, 'items')
items = removeduplicate(items)

def removeduplicate(it):
    # remove duplicates
    res_list = []
    for i in range(len(it)):
        if it[i] not in it[i + 1:]:
            res_list.append(it[i])
    return it

 

Any suggestions?

 

 

 

 

[WereCatf]

18 minutes ago, Shally said:

return it

Shouldn't you be returning res_list?

[BobVonBob]

23 minutes ago, Shally said:

items = local_request.extract_value(json_body, 'items')
items = removeduplicate(items)

def removeduplicate(it):
    # remove duplicates
    res_list = []
    for i in range(len(it)):
        if it[i] not in it[i + 1:]:
            res_list.append(it[i])
    return it

 

Any suggestions?

Is extract_value homemade? It's not a function I've seen before for dealing with JSON. What does it return?

Do you want to remove all but the first entry with a given itemID? All but the last? All of them?

With what I can guess, I have this advice.

 

You need to compare the itemID (depends on the format of extract_value), not the items themselves (it).

You append all unique items (not unique itemIDs) to res_list. Then you do nothing with res_list and return "it" unchanged.

Edited July 2, 2020 by BobVonBob
Last line italicized itself for some reason.

[WereCatf]

One could do this by e.g. this way:

import json

def dedupItems(items):
	dedupList = []
	dedupIDs = []
	for item in items["items"]:
		if(item["itemID"] not in dedupIDs):
			dedupList.append(item)
			dedupIDs.append(item["itemID"])
	return dedupList

#inJson should contain the JSON as a string that you wish to parse
items = json.loads(inJson)
items = dedupItems(items)

There are better ways, sure, but this is clear and understandable.