Theme 6: Qualitative and Case Study Research - Reflection

As this was the final theme for this course it was fitting to have course leader Leif Dahlberg holding the last seminar. There was a lot of focus on case studies since we have touched upon quality methods previous weeks, and we ended by briefly discussing some questions that came up regarding the course layout.

The seminar started off with us students dividing into smaller groups to discuss the different texts. Then the whole group started discussing together and most of the seminar went to talking about what a case study is and how it differs from a user study or a conventional research with a proper hypothesis. An object being studied in its natural habitat seemed to be a useful definition and an important thing is that the researchers are to take advantage of an existing situation rather than creating a scenario by themselves. From that point different types of qualitative and quantitative methods are welcomed in the study. I had not really considered the possibility of a case study prior to this week and I find it to be an interesting type of research method. When I did my bachelor's thesis a little over a year ago we were studying the possibility of incorporating Twitter in a creative performance process. We had an existing "case" were a dance company was developing a performance based off of Twitter and we sent out a survey for the users collaborating in the process and held interviews with the choreographers. All our research methods were held after the creative process had finished, so I guess that our project was a case study even though we did not consider this or even had knowledge of this being a certain method at the time.

Dahlberg also gave examples of "cases" being used in other situations, for example a police case or a medical case but also what is being used in economic studies: having a case, or a problem, that the student is supposed to solve. To wrap things up, I think that this week's theme was very interesting an I learned a lot about case studies and maybe a little bit about qualitative studies as well. It was a pleasant ending of this course and it was also good that we were able to ask some questions about the concluding blog post that is coming up after this one.

*We also had a postponed lecture with Haibo Li on Monday about design research and problem solutions. Li is a great lecturer (even though it can sometimes be somewhat difficult to understand) and I found his example from the film Johnny English to be different and a fun way to describe finding simple solutions. This lecture was a refreshing take on the process of defining, validating and evaluate your idea and the importance of defining your problem statement in order to find a solution.

Theme 6: Qualitative and Case Study Research

This is the last theme of the course and for this week I was to read three papers; two of my own choice that were using qualitative methods or the case study research method and one selected by the course teacher called Building Theories from Case Study Research by K.M. Eisenhardt. Aside from reading and reflecting on these papers and their methods I received the following tasks:

1. Briefly explain to a first year university student what a case study is.

2. Use the "Process of Building Theory from Case Study Research" (Eisenhardt, summarized in Table 1) to analyze the strengths and weaknesses of your selected paper.

Qualitative methods

I have chosen the article "Facebook and Online Privacy: Attitudes, Behaviors, and Unintended Consequences" written by Bernhard Debatin et al. It was published in the Journal of Computer-Mediated Communication in 2009 and discusses users' awareness of privacy issues on Facebook.They found that people associated the risks of privacy invasion to others rather than to themselves. They also understood that users reporting privacy invasion were more likely to change their privacy settings than those that had only heard about others’ privacy invasions and hadn't personally experienced this.

The authors started their research by sending out a quantitative online survey to 119 college students. As a result of the survey, 8 participants were chosen, based on their answers, for a qualitative and in-depth interview soon thereafter. The interviews were recorded, transcribed and then analyzed through a combination of tools such as qualitative content analysis and hermeneutical/rhetorical interpretation.

I find the selection method to be an interesting approach, but I feel that it can easily seem a little biased to choose the participants that are the most appropriate for the study. Of course this depends on what direction you would want to go regarding the research but I feel that to get the most honest result the interviewees should be randomized and their survey answers should not be reflected in the selection process. I think that this combination of qualitative and quantitative methods is very useful and it gives the researcher (and the reader) a wide foundation of results to base the conclusion on as it includes both general and specific findings. The analysis tools are things that I have no previous experience from and they sound very interesting to me. I was not aware that there are specific tools and methods for analyzing qualitative content and I thought that analyzing was something that the researchers had to do all by themselves without any guidelines. That is one of the most useful things I will bring from reading this paper and something I hope to have use of in my future studies.

Case study research
A case study can seem a lot like any research using qualitative methods on the first glance, but that is not necessarily the situation. When doing a case study one is observing an object in its natural habitat and analyzing real-life events. In order to do this combinations of qualitative and quantitative methods are often used and both empirical and theoretical theories are taken into account for the analysis.

For the case study research I have chosen to read the paper "Social media competitive analysis and text mining: A case study in the pizza industry" written by Wu He et al. It was published last year (2013) in the International Journal of Information Management. The paper takes a look on how pizza restaurants Pizza Hut, Domino's Pizza and Papa John's Pizza are using social media tools and how user-generated content can be used to their advantage. The writers of the paper are using a method called text mining to extract meaningful information from the user-generated data.

I find the paper to have more strengths than weaknesses and it follows the steps of Eisenhardt quite well. There are some research questions, but no hypothesis in order to retain the flexibility of the study. The selected cases are carefully chosen and the writers have a thorough enclosing comparison to other literature, both conflicting and agreeable. I think that the conclusion was short and sweet but I don't think that they found much other than that social media plays an important part in the relationship between restaurant and customer. I do however think that the study made a useful contribution for future research.

Theme 5: Design Research - Reflection

This week there were going to be two lectures with professors Haibo Li and Eva-Lotta Sallnäs, respectively. Unfortunately the first lecture was moved to the upcoming week instead and so I will only comment on the lecture with Sallnäs.

Sallnäs based the lecture on her own studies about haptics and how it can be implemented in different fields. I think that her intentions were that having an example of a certain research would make it easier for the students to get a grip on the general subject of design research but my takings of it was that it became a little too detailed and too much focus was put on Sallnäs' studies. Haptics is a very interesting subject and I think it could be used in many different areas other than educational purposes and medical research which seemed to be the more common areas for the moment. It could facilitate for global collaboration in many different perspectives and I find the development of augmented realities using responsive electronic tools to be intriguing.

Peas on Earth. Sallnäs mentioned the fun "fact" that peace on earth is a common reason for doing research! :D

The first thing Sallnäs did was was requesting the questions us students had prepared for the lecture. Only one student called out his question which I find disappointing. I do think, however, that the professor should have been more demanding that we all presented our questions because the initial silence was taken as us not having prepared for the lecture. I feel embarrased that I didn't call out my question which was "As there seem to be some research and guidelines regarding design research and qualitative methods, do researchers actually follow these guidelines?". The question was inspired by the previous theme as Olle Bälter mentioned that when it comes to quantitative methods the guidelines are not always followed. It would have been interesting to see a comparison of the two methods, but now I have learned that I must speak up in time or else I might miss my chance.

I'm looking forward to hear Li's lecture next monday which I hope to be interesting and informative!

Theme 5: Design Research

Part one of this theme was to read a paper by Haibo Li (et al) called Turn Your Mobile Into the Ball: Rendering Live Football Game Using Vibration and reflect on it with the help of some questions as follows:

1. How can media technologies be evaluated? 

2. What role will prototypes play in research? 

3. What are characteristics and limitations of prototypes? 

4. How can design research be communicated/presented? 

The second part included reading two texts co-penned by authors Eva-Lotta Sallnäs Pysander and Jonas Moll in 2013 and 2011 and reflect over the following questions: 

5. How does a collaborative setting differ from a single user setting as regards methodology used and the results obtained? 

6. How can qualitative and quantitative methods in the same study complement each other? 

7. How can using both subjective and objective methods give a better understanding of a phenomenon? 

My thoughts on the texts: 

Part 1:

When it comes to media technologies, enhancing the experience is key word for developing new ideas. Technology is evolving fast today and in order to keep that experience I believe that a functioning prototype and having the ability to test the usability are essential for proper evaluation. That is also the focus in the report, which is about using vibrations to render live football games.

Prototypes are important for testing the hypothesis of the research. When it comes to media technology a large part of researching is built on developing new products and new ways of using media and testing a concept is crucial to get any kind of result and being able to move forward in your research. Without prototypes explaining the concept to the user the research ill be unattainable. The main advantage of using a prototype is that the user gets a sincere sense of how a potential final product would behave. By using questionnaires and other theoretical methods the user can imagine what the product would be like but there's always going to be a limitation compared to testing a working prototype. With a prototype the user might present "live" reactions that they would not have considered when imagining a theoretical scenario. It is critical to remember that a prototype is not a finished product, or even a beta product, and is not fully developed. Users might be unhappy with the design and may, for example, give the prototype a lower and unrepresentative rating during user tests. Thus, the researcher should be clear about the differences that may occur to not deceive users and avoid misunderstandings.

Part 2:

Having a collaborative setting has its differences compared to a single-user setting and it depends on the situation whether they are advantages or disadvantages. When you have several test persons working in a collaborative setting the emotional data is going to differ from the single-user since the group will work together and one users' input to a problem might wake something in another person's mind that would not have come up in the single setting. I think working in a group (with satisfying group dynamics) will lead to solving the task in a more efficient and resourceful way and the participants will form a companionship towards each other but it will be difficult for the researcher to evaluate each individual process. In a single setting the process becomes more apparent but it may also be a bit limiting as previously mentioned since the user cannot contrive ideas from any co-users.

For a refined result within media technology that is general and fitting for a large target group, but at the same time can be cognitively analyzed, both qualitative and quantitative methods should me used. Larger quantitative studies such as questionnaires or simpler user tests (perhaps even online tests) will give the study a width, but I would say that it is also important to have advanced user tests and interviews that ca be analyzed in order to gain an understanding of the users and how they react and respond to the prototype. 

By looking objectively on a test person's attributes and their actions during a test and then combining this with subjective methods to learn about their experiences the researcher can gain more knowledge than by using only one or the other method. As mentioned in the previous section the combination is what will give the study a width and it will help the researcher to find a conclusion and find ideas for the development of the prototype.

Theme 4: Quantitative Research - Reflection

This week we only had a seminar held by Olle Bälter. The most interesting thing about the seminar was the set-up, which was a combination of some lecture slides and group activities. Bälter introduced a puzzle game called Boggle where the player is supposed to find as may words as possible by using a set of letters laid out in a matrix. Each word is worth one point, but only if your opponent hasn't found the same word. This form of playing was applied to the group challenges we received regarding quantitative research where we were to list as many advantages/weaknesses with, for example, quantitative methods or using web surveyes over paper surveys. We had three or four challenges and in the end the one group that had collected the most points was crowned the winner. I think this was a fun way to get our minds working and start thinking about the subject of different research metods and how they can be used. In the group I was in I feel that we tried really hard to achieve more points (and argue against the other groups getting their points!) but ultimately we landed only in third place out of four teams!

Bälter talked about how an optimal survey should be constructed. When asking a question where the answers ar scaled such as "agree completely, agree somewhat, disagree completely" there shoud always be an uneven number of alternatives so that there is always one neutral option such as "neither agree or disagree". Another tip wa that a statement should never include a subordinate clause that could negate the independent clause. An example of this is "some people play football, but I don't". the second clause is a neation and the test person may be confused on what they are supposed to answer to. The lecture about this subject were interesting. I have conducted a few questionnaires in previous courses but have not reflected on these things before other than trying to have as clear and understandable questions as possible. During this week's seminar I had a lot of fun, but I also learned a lot and obtained an understanding of quantative methods and their pros and cons. This theme as well as the last theme about theory I feel have been very useful in preparation for my upcoming master's thesis. While I thought the introductory themes were interesting I think that the course is taking a more relevant and concrete direction and I'm looking forward to the last two themes of the course.

Theme 4: Quantitative Research

The assignment for this week is a more open one than previous weeks. This time I am going to choose a research paper that is using quantitative methods in a good way and reflect on its use of research methods, Then I am going to read a paper written by guest lecturer Olle Bälter and reflect on the key points and what the text taught me. Two discussion questions follows the second text:

1. Which are the benefits and limitations of using quantitative methods?

2. Which are the benefits and limitations of using qualitative methods?

--- --- --- ---

The paper that I have chosen is called Measuring Mobile Phone Use: Self-Report Versus Log Data and was written by Jeffrey Boase in 2013. It examines the validity of self-report data to a log-based report system when it comes to the usage of cell phones. The method consisted of a questionaire allowing users to self-report their phone habits over the chosen period of time. This was then compared to data collected from the telecom operators and the writers of the report could analyze the reports made by the users to see how they percepted their own usage habits.

The writers admit to having some limitations with their research and list the geographical and cultural context of Norway (where the research took place) may not be representative for other societies. This is however a problem that most all studies suffer from; it is difficult to achieve a generaliziing result that is representative for any crowd other than the actual one being researched. It was mentioned that people who avoid such studies would possibly answer the survey differently from people who participate. Taking this a small step further, I can also imagine that people like to be percived as something they neccessarily are not and that they therefore "tweak" their answers to make themselves appear more sophisticated.

I found this paper interesting since I took part in a study regarding phone habits that some fellow classmates of mine did for their bachelor's thesis. I had to estimate the amount of time I spent with my cell phone and if I remember correctly it did not match the actual time very well. My own case is proof enough to me that self-reporting may not be as accurate as log-reporting, but I think that the combination that this report used is a good way to get a general perception of the usage.

The paper by Olle Bälter (et al) investigates a potential relationship between stress, physical health and Upper Respiratory Tract Infection, URTI. URTI includes diseases such as the flu and having a cold and it is one of the most common diseases in several countries. As a basis of the reserch quantitative online questionnaires were used where test persons belonging in the target group submitted information, since a previous study made by the authors had shown pleasing results using the same method.

1. Using quantitative methods, such as questionnaires, allows the researcher to collect a lot of data in a short period of time. The method also makes it easy for the test person since the questions often wants yes/no answers or just a brief commentary. A negative aspect of quantitative methods is that it is difficult to get any insight in the target group and the results will perhaps be generic.

2. Qualitative methods are beneficial because they provide inclusive data and the researcher gets to go in depth during, for example, an interview and ask questions that a form could not ask. Another good thing about qualitative methods is the fact that the answers are real and not some sort of average of the target group. However, a drawback from using these methods is that the result will be valid only for the specific case that provided the data and may not properly represent the complete research.

Theme 3: Research and Theory - Reflection

After attending the seminar this week I realize that my assumption regarding the type of theory was not exactly correct. Lecturer Leif Dahlberg explained the different types during the seminar which was helpful for trying to define the theory in a certain text. I looked back on the text I read about Khan Academy and now I would say that the theory of the text consists of a combination of the five types defined by Gregor. During the seminar this concept was something that came into discussion and Dahlberg argued that theories seldom are pure but rather a combination of several types. Worth mentioning, though, is that the five theories presented by Gregor aims to be extentions of each other; type II, explanation, is for example a continuation of type I, analysis, as analysis is one part of explanation as a whole.

This theme was a little more difficult than the previous two, but to me it had more relevant content and I feel like the things I have learned will be useful in future courses. One thing that stuck to me was the argument that studies are controlled by their theories and that the more developed a theory is, the easier it becomes to do a valuable study. This is very interesting and from experience I would have to agree with the statement. My best reports have been the ones where I have put much time and thought into the early stages, such as theory, of the process. I would say that theory can be seen as the foundation and prep work of a study, and if you don't have a proper foundation you cannot get a steady house.