This commit is contained in:
wea_ondara
2021-04-17 21:48:25 +02:00
parent cd2eff7bbf
commit f8e486b535
4 changed files with 106 additions and 11 deletions

View File

@@ -93,7 +93,7 @@ The onboarding process is a permanent challenge for online communities and diffe
One-day-flies are not unique to StackOverflow. \citeauthor{steinmacher2015social} investigated the social barriers newcomers face when they submit their first contribution to an open-source software project \cite{steinmacher2015social}. They based their work on empirical data and interviews and identified several social barriers preventing newcomers to place their first contribution to a project. Furthermore, newcomers are often on their own in open source projects. The lack of support and peers to ask for help hinders them. \citeauthor{yazdanian2019eliciting} found that new contributors on Wikipedia face challenges when editing articles. Wikipedia hosts millions of articles\footnote{\url{https://en.wikipedia.org/wiki/Wikipedia:Size_of_Wikipedia}} and new contributors often do not know which articles they could edit and improve. Recommender systems can solve this problem by suggesting articles to edit but they suffer from the cold start problem because they rely on past user activity which is missing for new contributors. \citeauthor{yazdanian2019eliciting} proposed a solution by establishing a framework that automatically creates questionnaires to fill this gap. This also helps matching new contributors with more experienced contributors that could help newcomers when they face a problem.
\citeauthor{allen2006organizational} showed that the one-time-contributors phenomenon also translates to workplaces and organizations \cite{allen2006organizational}. They found out that socialization with other members of an organization plays an important role in turnover. The better the socialization within the organization the less likely newcomers are to leave. This socialization process has to be actively pursued by the organization.
One-day-flies may partially be a result of lurking. Lurking is consuming content generated by a community but not contributing content to it. \citeauthor{nonnecke2006non} investigated lurking behavior on Microsoft Network (MSN) \cite{nonnecke2006non} and found that contrary to previous studies lurking is not necessarily a bad behavior. Lurkers show passive behavior and are more introverted and less optimistic than actively posting members of a community. Previous studies suggested lurking is free riding, a taking-rather-than-giving process. However, the authors found that lurking is important in getting to know a community, how a community works and learning the nuances of social interactions on the platform. This allows for better integration into the community when a person decides to join the community. StackExchange, and especially the StackOverflow community, probably has a large lurking audience. Many programmers do not register on the site and those who do only ask one question and revert to lurking, as suggested by \cite{slag2015one}.
One-day-flies may partially be a result of lurking. Lurking is consuming content generated by a community but not contributing content to it. \citeauthor{nonnecke2006non} investigated lurking behavior on Microsoft Network (MSN) \cite{nonnecke2006non} and found that contrary to previous studies \cite{kollock1996managing, morris1996internet} lurking is not necessarily a bad behavior. Lurkers show passive behavior and are more introverted and less optimistic than actively posting members of a community. Previous studies suggested lurking is free riding, a taking-rather-than-giving process. However, the authors found that lurking is important in getting to know a community, how a community works and learning the nuances of social interactions on the platform. This allows for better integration into the community when a person decides to join the community. StackExchange, and especially the StackOverflow community, probably has a large lurking audience. Many programmers do not register on the site and those who do only ask one question and revert to lurking, as suggested by \cite{slag2015one}.
% DONE Non-public and public online community participation: Needs, attitudes and behavior \cite{nonnecke2006non} about lurking, many programmers do that probably, not even registering, lurking not a bad behavior but observing, lurkers are more introverted, passive behavior, less optimistic and positive than posters, prviously lurking was thought of free riding, not contributing, taking not giving to comunity, important for getting to know a community, better integration when joining
@@ -136,6 +136,36 @@ Unwelcomeness is a large problem on StackExchange \cite{ford2016paradise}\footre
\subsection{Keeping users engaged, contributing and well behaved}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%new
%https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.592.1587&rep=rep1&type=pdf \cite{iriberri2009life}
% -> about community life cycle, systainablity; READMORE cap 5&6&*8*&10.3&10.4&10.5
% -> look at success factors in table IX and X
% -> look at refs
% -> look at how to integrate that with kraut etal
% look for parallels between papers and stackoverflow and write somethings about how stack overflow does it
% split into growth and sustainablity capters (maybe, depends on how well i can be split)
% IMPORTANT: recognize user contributions, with goodies \cite{iriberri2009life}
% community management (social managment)
% -> voluntarism
% -> reasons user would do that: altruistic(do good for the community), or selfish reasons (recognition from others (superiors), promotions, etc.) \cite{ginsburg2004framework}
% -> even if community is lead by paid employees, volunteers to most of the community work \cite{butler2002community}
% -> important factors: trust, reputation, identity \cite{ginsburg2004framework}
% other studies which suggest changes to improve community interaction/qualtity/sustainability
% -> help vamipires, noobs, reputation collectors \cite{srba2016stack}
% -> qualtity solution suggestions \cite{srba2016stack}
% -> restrict openness of the community, not desirable (e.g. restrict number of questions to combat low-quality questions), will not be 100% efective\cite{srba2016stack}
% -> ''Improving Low Quality Stack Overflow Post Detection`` \cite{ponzanelli2014improving}, reduce review queue for moderators
% -> finding content abusers, yahoo answers \cite{kayes2015social}, other communities \cite{cheng2015antisocial}
% -> matching questions with answerers \cite{srba2016stack} (difficult questions -> expert users, easier questions -> answerers that know it but are not experts), dont overload experts, utilize capacities of the many nonexperts
% TODO look if moderation features are covered
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%intro .. se employes serveral features to engage/keep contributing users
%reputation
%badge system
@@ -173,9 +203,9 @@ Different badges also create status classes \cite{immorlica2015social}. The hard
Quality is often a concern in online communities. Platform moderators and admins want to keep a certain level of quality or even raise it. However, higher-quality posts take more time and effort than lower-quality posts. In the case of CQA platforms, this is an even bigger problem as higher quality answers fight against fast responses. Despite that, StackOverflow also has a problem with low quality and effort questions and subsequent unwelcoming answers and comments\footref{silge2019welcome}. StackOverflow has grown into a large community and larger communities are harder to control. \citeauthor{lin2017better} investigated how growth affects a community. They looked at Reddit communities that were added to the default set of subscribed communities of every new user (defaulting) which lead to a huge influx of new users to these communities as a result. The authors found that contrary to expectations, the quality stays largely the same. The vote score dips shortly after defaulting but quickly recovers or even raises to higher levels than before. The complaints of low-quality content did not increase, and the language used in the community stayed the same. However, the community clustered around fewer posts than before defaulting.
Quality is often a concern in online communities. Platform moderators and admins want to keep a certain level of quality or even raise it. However, higher-quality posts take more time and effort than lower-quality posts. In the case of CQA platforms, this is an even bigger problem as higher quality answers fight against fast responses. Despite that, StackOverflow also has a problem with low quality and effort questions and subsequent unwelcoming answers and comments\footref{silge2019welcome}. StackOverflow has grown into a large community and larger communities are harder to control. \citeauthor{lin2017better} investigated how growth affects a community. They looked at Reddit communities that were added to the default set of subscribed communities of every new user (defaulting) which lead to a huge influx of new users to these communities as a result. The authors found that contrary to expectations, the quality stays largely the same. The vote score dips shortly after defaulting but quickly recovers or even raises to higher levels than before. The complaints of low-quality content did not increase, and the language used in the community stayed the same. However, the community clustered around fewer posts than before defaulting. \citeauthor{srba2016stack} did a similar study on the StackOverflow community \cite{srba2016stack}. They found similar pattern in the quality of posts. The quality of questions dipped momentarily due to the huge influx of new users. However, the quality did recover after 3 months. They also identified 3 types of users causing the lowering of quality: \emph{Help Vampires} (these spend litte to no effort to research their questions, which leads to many duplicates), \emph{Noobs} (they create mostly trivial questions), and \emph{Reputation Collectors} (they try to gain repuation as fast as possible by methods described by \citeauthor{bosu2013building}\cite{bosu2013building} but often with no reguard of what effects their behavior has on the community, for instance, encouraging \emph{Help Vampires} and \emph{Noobs}).
\citeauthor{tausczik2011predicting} found reputation is linked to the perceived quality of posts in multiple ways \cite{tausczik2011predicting}. They suggest reputation could be used as an indicator of quality.
Quality also depends on the type of platform. \cite{lin2017better} showed that expert sites who charge fees, for instance, library reference services, have higher quality answers compared to free sites. Also, the higher the fee the higher the quality of the answers. However, free community sites outperform expert sites in terms of answer density and responsiveness.
Quality also depends on the type of platform. \citeauthor{lin2017better} showed that expert sites who charge fees, for instance, library reference services, have higher quality answers compared to free sites\cite{lin2017better}. Also, the higher the fee the higher the quality of the answers. However, free community sites outperform expert sites in terms of answer density and responsiveness.
% quality

View File

@@ -18,15 +18,11 @@ StackExchange introduced a \emph{new contributor} indicator to all communities o
This thesis investigates the following criteria to determine whether the change affected a community positively or negatively, or whether the community is largly unaffected:
\begin{itemize}
\item \textbf{Sentiment of answers to a question}. This symbolizes the quality of communication between different individuals. Better values indicate better communication. Through the display of the \emph{new contributor} indicator, the answerer should react less negatively towards the new user when they behave outside the community standards.
\item \textbf{Vote score of questions}. This is similar to the sentiment criterion. Voters will likely vote more postively (not voting instead of down voting, or upvoting instead of of not voting) due to the \emph{new contributor} indicator. Thereby the vote score should increase after the change.
\item \textbf{The amount of first and follow-up question}. This symbolizes the willingness of users to participate in the community. Higher amounts of first questions indicate higher number of new participating users. Higher follow-up questions indicate that users are more willing to stay within the community.
\item \textbf{Vote score of questions}. This symbolizes the feedback the community give to a question. Voters will likely vote more postively (not voting instead of down voting, or upvoting instead of of not voting) due to the \emph{new contributor} indicator. Thereby the vote score should increase after the change.
\item \textbf{The amount of first and follow-up question}. This symbolizes the willingness of users to participate in the community. Higher amounts of first questions indicate higher number of new participating users. Higher follow-up questions indicate that users are more willing to stay within the community and continue their active participation.
\end{itemize}
If these criteria improve after the change is introducted, the community is affected positively. If they worsen, the community is affected negatively. If the criteria stay largely the same, then the community is unaffected. Here it is important to note that a question may receive answers and votes after the \emph{new contributor} indicator is no longer shown and therefore these are not considered as part of the data set to analyze.
%only when new contributor insicator is shown
%limitations
% large sudden changes (maybe include example from analysis)
% autocorrelation?
% TODO
@@ -35,7 +31,7 @@ To measure the effect on sentiment of the change this thesis utilizes the Vader\
However, just simply looking at the words in a text is not enough and therefore Vader also uses rules to determine how words are used in conjunction with other words. Some words can boost other words. For example, ``They did well.'' is less intense than ``They did extremely well.''. This works for both positive and negative sentences. Moreover, words can have different meanings depending on the context, for instance, ``Fire provides warmth.'' and ``Boss is about to fire an employee.'' This feature is called \emph{Word Sense Disambiguation}.
Furthermore, Vader also detects language features commonly found in social media text which may not be present in other forms of text, for instance, books, or news papers. Social media texts may contain acronyms, initialisms (for instance \emph{afaik} (as far as I know)), slang words, emojis, caps words (often used to emphasize meaning), punctuation (for instance, \emph{!!!}, and \emph{?!?!}), etc.. These features can convey a lot of meaning and drastically change the sentiment of a text.
After all these features are considered, Vader outputs a sentiment value between -1 and 1 on a continuous range. The sentiment range is divided into 3 classes: negative (-1 to -0.05), neutral (-0.05 to 0.05), and positive (0.05 to 1). The outer edges of this range are rarely reached as the text would have to be extremely negative or positive which is very unlikely.
After all these features are considered, Vader assigns a sentiment value between -1 and 1 on a continuous range. The sentiment range is divided into 3 classes: negative (-1 to -0.05), neutral (-0.05 to 0.05), and positive (0.05 to 1). The outer edges of this range are rarely reached as the text would have to be extremely negative or positive which is very unlikely.
%speed
Due to this mathematical simplicy, Vader is really fast when computing a sentiment value for a given text. This feature is one of the requirements \citeauthor{hutto2014vader} originally posed. They proposed that Vader shall be fast enough to do online (real time) analysis of social media text.
@@ -79,6 +75,12 @@ After preprocessing the raw data, relevant data is filtered and computed. Questi
An interrupted time series (ITS) analysis captures trends before and after a change in a system and fits very well with the question this thesis investigates. ITS can be applied to a large variety of data if the data contains the same kind of data points before and after the change and when the change date and time are known. \citeauthor{bernal2017interrupted} published a paper on how ITS works \cite{bernal2017interrupted}. ITS performes well on medical data, for instance, when a new treatment is introduced ITS can visualize if the treatment improves a condition. For ITS no control group is required and often control groups are not feasible. ITS only works with the before and after data and a point in time where a change was introduced.
ITS relies on linear regression and tries to fit a three-segment linear function to the data. The authors also described cases where more than three segments are used but these models quickly raise the complexity of the analysis and for this thesis a three-segment linear regression is sufficient. The three segments are lines to fit the data before and after the change as well as one line to connect the other two lines at the change date. Figure \ref{itsexample} shows an example of an ITS. Each segment is captured by a tensor of the following formula $Y_t = \beta_0 + \beta_1T + \beta_2X_t + \beta_3TX_t$, where $T$ represents time as a number, for instance, number of months since the start of data recording, $X_t$ represents 0 or 1 depending on whether the change is in effect, $\beta_0$ represents the value at $T = 0$, $\beta_1$ represents the slope before the change, $\beta_2$ represents the value when the change is introduced, and $\beta_3$ represents the slope after the change. Contrary to the basic method explained in \cite{bernal2017interrupted} where the ITS is performed on aggregated values per month, this thesis performs the ITS on single data points, as the premise that the aggregated values all have the same weight within a certain margin is not fulfilled for sentiment and vote score values. Performing the ITS with aggregated values would skew the linear regression more towards data points with less weight. Single data point fitting prevents this, as weight is taken into account with more data points. To filter out seasonal effects, the average value of all data points with the same month of all years is subtracted from the data points (i.e. subtract the average value of all Januaries from each data point in a January). This thesis uses the least squares method for regression.
Although, the ITS analysis takes data density variability and seasonality into account, there is always a possibility that an unknown factor or event is contained in the data. It is always recommended to do a visual inspection of the data. This thesis contains one example where the data density increases so drastically in a particular time span that this form of analysis looses accuracy.
%limitations
% large sudden changes (maybe include example from analysis)
% autocorrelation?
%
\begin{figure}
\centering\includegraphics[scale=0.7]{figures/itsexample}

View File

@@ -362,3 +362,66 @@
year={2019},
publisher={Oxford University Press}
}
@article{iriberri2009life,
title={A life-cycle perspective on online community success},
author={Iriberri, Alicia and Leroy, Gondy},
journal={ACM Computing Surveys (CSUR)},
volume={41},
number={2},
pages={1--29},
year={2009},
publisher={Acm New York, NY, USA}
}
@article{kollock1996managing,
title={Managing the virtual commons},
author={Kollock, Peter and Smith, Marc},
journal={Computer-mediated communication: Linguistic, social, and cross-cultural perspectives},
pages={109--128},
year={1996},
publisher={John Benjamins Philadelphia, PA}
}
@article{morris1996internet,
title={The Internet as mass medium},
author={Morris, Merrill and Ogan, Christine},
journal={Journal of Computer-Mediated Communication},
volume={1},
number={4},
pages={JCMC141},
year={1996},
publisher={Oxford University Press Oxford, UK}
}
@article{butler2002community,
title={Community effort in online groups: Who does the work and why},
author={Butler, Brian and Sproull, Lee and Kiesler, Sara and Kraut, Robert},
journal={Leadership at a distance: Research in technologically supported work},
volume={1},
pages={171--194},
year={2002},
publisher={Taylor \& Francis New York}
}
@inproceedings{ginsburg2004framework,
title={A framework for virtual community business success: The case of the internet chess club},
author={Ginsburg, Mark and Weisband, Suzanne},
booktitle={37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the},
pages={10--pp},
year={2004},
organization={IEEE}
}
@article{srba2016stack,
title={Why is stack overflow failing? preserving sustainability in community question answering},
author={Srba, Ivan and Bielikova, Maria},
journal={IEEE Software},
volume={33},
number={4},
pages={80--89},
year={2016},
publisher={IEEE}
}
@inproceedings{ponzanelli2014improving,
title={Improving low quality stack overflow post detection},
author={Ponzanelli, Luca and Mocci, Andrea and Bacchelli, Alberto and Lanza, Michele and Fullerton, David},
booktitle={2014 IEEE international conference on software maintenance and evolution},
pages={541--544},
year={2014},
organization={IEEE}
}

2
todo2
View File

@@ -11,7 +11,7 @@
3.
- DONE argumente warum ich genau diese variablen (sentiment, votes, #questions)
- DONEXT limitierungen, andere faktoren
- DONE MAY LOOK AGAIN limitierungen, andere faktoren
- DONE vader genau beschreiben
5.