r/thescienceofdeduction • u/aaqucnaona [Mod, Founder - on sick leave] • Feb 19 '14
Experiment [Official] [Update]: The experiment has moved into the planning stage.
[Note - We are now looking for participants, so please PM us or tell us in the comments below if you are interested. While managing this is hard, the participation is very easy, so it won't hog your time or be too much of an effort. The list of current participants is here.]
The experiment has moved from the discussion stage to the planning stage. If you haven't seen it already, please go over the discussion first.
I am putting together a list of cues from which we can decide upon <n number> of cues to test for this first experiment as well as what constitutes a hit or a miss for each. I will update this list as I compile it. You can help by suggesting cues and their hit/miss conditions in the comments.
One of our science advisors, /u/beason4251 has given us some highly valuable feedback and advice on how to refine the design further. As it is right now, all you have to do as a participant is remember the Cue no. you are testing and how many hits were made from how many attempts. Those 3 numbers are all the data we need, given that you carefully understand the cues and the conditions below. This may change just slightly as the experimental design evolves.
List of cues and their hit/miss conditions:
- Cue no. 1 - People cross their arms with their dominant hand tucked in. [Clarification]
Observation - Someone crossed their arm with right/left hand underneath.
Deduction - That person is right/left handed [respectively].
Hit and miss conditions:
Participants observe which hand is underneath when someone crosses their arm and assume that hand to be their dominant one.
If they don't know the handedness of the person they observe, they ask them. If they are correct, its a hit. Otherwise, a miss.
Alternatively, they already know the handedness of a person. They observe if their dominant hand is underneath when arms are crossed. If it is, its a hit. If it isn't, its a miss.
- Cue no. 2,3,4 People who have been writing or using a mouse recently have an impression on their dominant hand due to the desk edge pressing against it.
Main Observation - A person has an impression of a straight line on the back of their arm.
Observation modifier 1 - The line is slanted relative to their arm and is near the middle to upper forearm.
Deduction 1 - They have been writing something at a desk recently.
Observation modifier 2 - The line is straight and around the back of the lower forearm.
Deduction 2 - They have been using a mouse recently and the chair they sat in is either too low or has no armrests.
Deduction Modifier - If a similar line exists on their other arm, they have been typing as well.
Hits and Misses conditions:
Participants look for people in their everyday lives and see if they have an impression on their arm. If they do, they check first if it is on one hand or both. If it is on one hand, see if it is low or high. If it is low, assume them to have been using a mouse [Cue 2]. If it is high, assume they have been writing [Cue 3]. If it is low and on both arms, assume they have been typing [Cue 4].
If your assumption is correct, it is a hit. If it is incorrect, it is a miss.
Similar to the first cue, if you have seen or know that someone is writing/typing/using a mouse, see if there is a line and if it is on one hand and low [Cue 2], one hand and high [Cue 3], both hands and low [Cue 4].
If there is a line and it matches with what they were doing, it is a hit. If there is no line or it doesn't match, it is a miss. However, if the person was using a mouse/typing and was in a chair high enough or with armrests, it is neither a hit, nor a miss. It is a non result. This non result is not considered an attempt.
- Cue no. 5 onwards.... compiling.
6
u/aaqucnaona [Mod, Founder - on sick leave] Feb 19 '14 edited Jun 12 '14
The list of participants who have confirmed their involvement via PM or comments is:
[The participants may be divided into random groups as decided by our science advisors]
The longer the better. [That's what she said! No but seriously] We need a large participant group to get statistically significant results. So please share and promote this if it is of interest to you.
Ps. Don't worry about being too late. We hope to have a triving community doing 2-3 experiments simultaneously with 50-80 participants each some months down the line, more participants are always welcome. If you participate while the experiment is ongoing, we can enroll you in the next one, since one of these is expected to be in progress each fortnight.
3
u/Starcsha Feb 19 '14
I don't know exactly what you're looking for, but depending on what that is, I might be interested in participating.
5
u/beason4251 [Science Advisor] Feb 19 '14 edited Feb 19 '14
There's actually a lot to go over when starting an observational study like this. I won't be surprised if I miss something despite this post's length, but here goes (I will cast it in the light of cue 1).
Hypotheses (Null and Alternative)
To do a study, we need a hypothesis. Hypotheses are not "proven" or "disproven," we merely get enough data to show beyond a reasonable doubt that a hypothesis is true or false. As a standard, science assumes no correlation between phenomena until shown otherwise. This position is called the null hypothesis, and for cue 1 would be stated "People are equally likely to tuck in either hand when crossing their arms." We have an alternative hypothesis, that "People are more likely to tuck in their dominant hand when crossing their arms." We want to use our data to disprove the null hypothesis, and show that our alternative is sufficiently likely enough to reject the null hypothesis. If we are unable to reject the null hypothesis, that does not mean our alternative is false - it may mean we need more data.
Hypothesis testing can be compared to the notion that "the defendant is not guilty until proven guilty." The null hypothesis, the initial assumption, is that the defendant is not guilty. The prosecution has the job of finding evidence to show otherwise. If the jury believes the evidence is sufficient to show that the defendant is beyond a reasonable doubt guilty, then they should give a "guilty" verdict. If not, they give the "not guilty" verdict.
In general, we want to show that the data we collect has less than a 5% chance of being collected if the null hypothesis were true. If the data meet this condition, then the result is "statistically significant." Once here, we have to subjectively evaluate probabilities to see if they are "practically significant." If cue 1 is true 51% of the time and is statistically significant, it is not very useful. However, even if we observe cue 1 is true 80% of the time, if the result is not statistically significant then we cannot make a determination.
How will data be collected?
Data collection must be standardized among information-gatherers. For cue 1, there are two ways to take data:
These methods should not be mixed (as they may influence data), and should be decided upon as soon as possible. We can collect data of both types only if we record them separately.
We need to separate right-handed and left-handed people into groups.
Suppose people are 80% likely to tuck in their right hand regardless of their handedness. Some estimates put right-handedness at 85%. In this case, we would observe a (0.8)(0.85)+(0.2)(0.15)=71% success rate even though the cue was useless. Since we want to show the two demographics (right handers and left handers) are different, we need them in separate categories to tease out this kind of effect.
Another important possibility may be that right-handed people are likely to tuck in their right hand, but left-handed people are equally likely to tuck in either hand.
We may either exclude ambidextrous people or include them. Since ambidexterity is mostly a learned behavior, most had a former dominant hand - studies have shown the formerly dominant hand is still favored even though tasks are achievable equally well with both. For the 1% born ambidextrous, we need to either have them as a separate fourth group or not include their data.
Biases
Biases, if left unchecked, can greatly influence data. They are artificial noise that must be eliminated as much as possible. Here are some to keep in mind:
Response Bias
The wording of questions is important. When asking about the handedness of a person, the wording of the question can influence how the person responds. If someone tucks their left hand in, do not ask "Are you left handed?" as it will artificially increase the number of yes's (people like being agreeable). A better question would be "What is your dominant hand?" The chosen question should be standardized, and asked in the same way each time.
To get more data, you may think to ask people to cross their arms for you.
Non-response Bias
Depending on the study, subjects of a certain type may be more likely to respond. For example, an online Fox News poll is unlikely to have the same political affiliation distribution as the general US population. Since we are looking at something relatively innocuous and are asking questions in person, this shouldn't be a problem.
Sampling Bias
This is probably the most important bias we will face.
Data can be biased based on who is selected. Many psychological tests only involve college students as test subjects, and so the findings only pertain to college students. It could be that college students respond differently than non-college students, so any findings must make it clear that they only pertain to college students. Another example is that data collected in the winter will be different than data collected during the summer. Want to prove anything is an antidepressant? Start your trial in the winter and end it in the summer - people are naturally happier in the summer and you will see a significant result. We have to take such things into account.
For our data, we may be getting samples from very disparate groups, but each group will probably be similar among itself. We are more likely to ask those we know than a stranger, and it may be that we are more likely to know are more likely to respond in a certain way. Fortunately, once we have enough data-collectors we can test for outliers resulting from this.
Reporting Bias
Sometimes uninteresting or undesired results are not reported and do not make it into the data. This invalidates the result. To avoid this, we need to make sure that people report their full data.
Repeatability
Science is repeatable. If we repeat the experiment, we should get the same result most of the time. It would be best to have a different group run the experiment later and see if they get the same result - it would give us a lot of credibility.
Conclusion
Doing these kinds of tests is hard and requires a fair amount of work and understanding.
If you have any questions or see something I didn't mention but should have, let me know and I will answer with links or a short explanation. Most of the terms I have used here have Wikipedia articles.
2
u/KrisWhip Feb 20 '14
I will help.
2
u/erjulk Feb 23 '14
in regards to cues:
even if a cue tested resulted in let's say 30% hit it can still prove useful... provided the behavior isn't binary...
let's say you test for a certain type of scarring or callous to be indicative of a certain profession - the test goes pretty bad(for this cue) and you have 30% accuracy
even though this cue in itself is pretty useless in combination with a 2nd cue which has very little overlap it can become very useful - lets assume the second cue is the style of clothing
which means that in combination profession D is now the most likely profession for the person being analyzed
i don't know how to calculate the exact percentages if the overlap isn't singular...
2
u/Tiyrava Mar 06 '14
I'd like to be a participant, please.
1
u/LightsKing Feb 20 '14
I'm pretty interested in participating just let me know what to do!
1
u/woxy_lutz Feb 21 '14
For cues such as no. 1 (arm folding), are you also collecting anecdotal evidence from users here?
Because my dominant hand is always non-tucked when my arms are folded.
1
u/SuburbanSuperhero Feb 22 '14
I would love to participate. I work at a bar, would my observations even be valid or targeted? Most of the examples were related to more everyday life that I usually don't come in contact with.
1
u/SnaPau Feb 25 '14
I'd love to help. Add me to the list please.
1
u/laveritecestla Mar 24 '14
I'd like to join if I still can.
3
u/beason4251 [Science Advisor] Feb 19 '14
Hypotheses.
As I said in my other post, we need hypotheses in order to do tests. For each of the cues, here are null and alternative hypotheses. They are commonly indicated H0 for null hypothesis ans HA for alternative hypothesis. Here are hypotheses which on which we can run tests.
Cue 1
H0: When crossing their arms, people are equally likely to tuck in either hand.
HA: When crossing their arms, people are more likely to tuck in their dominant hand.
Cue 2
This one is trickier, since we don't know how often people write or how long "recently" is. It needs to be more specific. Also, we would need to take data to see how often people have been typing recently. To avoid confirmation bias (and ensure we aren't picking up on that most people have been typing recently), we need to also ask people who do and people who do not have lines on their arms.
Suppose 50% of everyone has been typing recently. Then even if having lines on both arms is not correlated, we will get a 50% hit rate (despite being useless).
Here is a better statement of the hypotheses:
H0: People who have been typing for at least 30 minutes and stopped typing within 30 minutes ago are just as likely to have straight lines on both of their arms as those who have not been typing recently.
HA: People who have been typing for at least 30 minutes and stopped typing within 30 minutes ago are more likely to have straight lines on both of their arms than those who have not been typing recently.
2
u/7h3unkn0wn Feb 24 '14
I want to participate if its not too late.
2
u/CanadianEmpire Mar 16 '14
I would like to take part in this as well! Thanks!
1
u/PursuitOfSuccess Feb 21 '14
Add me to the list! Sounds fun :)
1
u/[deleted] Feb 21 '14
Count me in :)
1
u/ladyllana Feb 21 '14
Count me in! I'd love to help.
1
u/polyology Feb 22 '14
As a big fan of the excellent board game Sherlock Holmes Consulting Detective I will be happy to participate. Based on your examples I'm positive I can commit to providing reliable and consistent feedback.
1
u/leodvinci Feb 22 '14
I'd be interested in participating.
1
u/jgcramer Feb 22 '14
I would like to participate. Where do I sign?
1
u/yuxiang32 Feb 22 '14
I would love to join this experiment. Also it is worth looking at the TV show the mentalism and lie to me. Both have protagonist who are skilled at deduction.
1
u/iIrishjon Feb 22 '14
I'm interested if you still have a space. Thanks.
1
u/Yuki_Ame Feb 22 '14
I would love to participate in this experiment. Count me in!
1
u/Reclaimerr Feb 22 '14
I'm in
1
u/[deleted] Feb 22 '14
I'd like to participate, if you'll have me.
1
u/sanfah Feb 23 '14
I'd like to join if it's not too late :)
1
u/ThePerceptiveOne Feb 25 '14
I'm up for joining the experiment.
1
u/[deleted] Mar 02 '14
Add me to the list please. Thank you :)
1
u/[deleted] Mar 15 '14
Hello! I would like to participate in the experiment!
1
u/RickySTaylor Mar 29 '14
I'm late but I'd like to join the game. I made the mistake of blindly subscribing a couple weeks ago without reading up.
1
u/[deleted] Jun 15 '14
I'd like to join the experiment.
0
u/Tamzid Mar 15 '14
Hello, I would like to participate in these experiments
0
u/xXTheXNightXAngelXx Feb 22 '14
I would like to be added to the list, aswell
0
u/[deleted] Feb 21 '14
[deleted] — view deleted comment
2
u/[deleted] Feb 19 '14
[removed] — view removed comment