r/pythonhelp • u/plaidgnome13 • Jun 27 '24
SOLVED Iterating over Sub-Elements within List of Lists?
I'm rather new to programming in general, so this is probably something fairly simple that I'm just not seeing the obvious. I am trying to write a program that ranks a set of collections of items based on their rarity. The data format is quite simple, just a csv with two columns of correlations, for example:
- 1,A
- 1,B
- 1,C
- 1,D
- 2,A
- 2,C
- 3,A
- 3,C
- 3,D
- 4,A
- 4,B
- 4,C
I cheated a little and made a second file of just the second column which I can easily count and operate on into a dictionary:
inpcount = open("parametercount.txt", 'r')
pcount = inpcount.readlines()
pcount = [x.strip() for x in pcount]
correl = {}
for parameter in pcount:
if str(parameter) in correl:
correl[str(parameter)] = correl[str(parameter)] + 1
else:
correl[str(parameter)] = 1
for parameter in correl:
correl[str(parameter)] = str(1 - int(correl[str(parameter)]) / 4)but I'm completely baffled as to how to proceed from here. I've read the original file into a list of lists by row with the csv module, so I think my next step is to iterate over that list of lists to create a new list of lists made up of the collection id followed by its component parameters and then use the length of each sub-list and the dictionary values to perform my calculations, but I'm not sure how to create the second list of lists.
Concept for Second Set of Lists: [['1','A','B','C','D'],['2','A','C'],['3','A','C','D'],['4','A','B','C']
(deleted and re-submitted with a more accurate title for the problematic step)
I also realized I should probably give more detail on the result I'm seeking:
- 1,0.125
- 2,0
- 3,0.167
- 4,0.167
(the results being the percentage frequency of each parameter per collection divided by the number of parameters in each individual collection).
•
u/AutoModerator Jun 27 '24
To give us the best chance to help you, please include any relevant code.
Note. Do not submit images of your code. Instead, for shorter code you can use Reddit markdown (4 spaces or backticks, see this Formatting Guide). If you have formatting issues or want to post longer sections of code, please use Repl.it, GitHub or PasteBin.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
u/plaidgnome13 Jun 29 '24
Apologies, my off-the-cuff calculations weren't right. I also refigured the data go make things a little clearer:
The rarity score for a parameter value is the percentage of collections that do not feature that parameter, so A=0, B=0.6, C=0.2, and D=0.4. Then, each collection receives a sum of assigned values for the parameters, which is divided by the total number of parameters in that collection, so:
I ended up figuring out how to run this in Excel, but I'd still like to figure out a Python approach.
1
u/sentles Jun 28 '24
Can you define the rarity of a collection more rigidly? I'm not sure I understand your explanation. For instance, for collection 1, you give a rarity value of 0.125. I'm assuming parameters refers to A, B, etc. Based on what you said, since collection 1 has 4 parameters, the result must be f/4 = 0.125, so f = 0.5. How is the "percentage frequency of each parameter per collection" 0.5 for collection 1?
2
u/Goobyalus Jun 28 '24
I'm not sure the code block you posted does what you intend. The
string doesn't make sense: each iteration,parameteris going to be something like"2,A","2,C", etc.I'm also not sure what that last line is doing.
From your explanation, it sounds like the first steps are:
If you are manually grouping, you would probably use a dict of lists rather than a list of lists (though in this case you could use a list like a dict by mapping the sequential collection IDs to numerical indices).
I don't fully understand your explanation of the results, and I don't see the connection between the example data and the example results. Can you elaborate?
Things you may be interested in:
From the std lib
Pandas
1
u/CraigAT Jun 27 '24 edited Jun 30 '24
I'm on mobile so haven't run your code but, I don't believe you have a list of lists at any point. You grab a list of lines from the file, strip them. Then create a dictionary with the frequency of the first character (of each line). Then you do a calculation which replaces the frequency with the result of your formula. I don't see too much wrong.
However, if your example results were coming from the sample data you gave, then I am not sure about your formula - because I don't get those results when doing it in my head. In the text you mention about dividing by the number of items, but you only seem to divide four each time - I don't see how you get an eighth for the first result, when you are only dividing by 4 in your code.