304 lines
21 KiB
TeX
304 lines
21 KiB
TeX
\chapter{Results}
|
|
|
|
%TODO some text here
|
|
This section shows the results of the experiments described in section 3 on the data sets described in section 4. In the following pages, there 3 diagrams for each community.
|
|
|
|
In diagrams (a), the blue line states the average sentiment (\emph{average sentiment} in diagram legend) of the answers to questions from new contributors. Also, the numbers attached to the blue line indicate number of answers to questions from new users that formed the average sentiment. The orange line (\emph{sm single ITS} in the diagram legend) represents the ITS over the whole period of the avaiable data. As stated in section 3.2, data density variabilty is a factor to take into account, therefore, the orange line represents the weighted ITS. The green, red, purple, and brown lines also represent ITS, however the time period considered for ITS before and after the change are limited to 6, 9, 12, and 15 months respectively.
|
|
|
|
Similarly, in diagram (b), the blue line respresens the average vote score of the questions of new users. The number attached to the blue line indicate the number of questions that formed the average vote score. The ITS (orange, green, red, purple, and brown lines) are computed the same way as in diagrams (a).
|
|
|
|
In diagrams (c), the blue line represents the number of 1st questions from new users, whereas the orange line denotes the followup questions from new users. The green and red lines represent the ITS of the blue and orange line respectively. In these diagrams no weighting is performed as each data point has equivalent weight.
|
|
|
|
|
|
% pvalues ...
|
|
% maybe average data points per month
|
|
|
|
|
|
|
|
\pagebreak
|
|
\section{StackOverflow.com}
|
|
StackOverflow shows a very slight decrease in the average sentiment of time before the change is introduced. When the change occurs the average sentiment jumps up. After the change, the sentiments reach higher levels and keep rising. The average vote score rises right before and stays fairly constant after the change. This indicates that the vote score is not affected by the change. However, the number of questions from new contributors increases after the change while before the change is fairly constant. The number of follow-up questions from new contributors declines before the change and rise after the change.
|
|
\begin{figure}[H]
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../stackoverflow.com/output/its/average_sentiments-i1.png}
|
|
\caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on StackOverflow.com}
|
|
\label{stackoverflow_its}
|
|
\end{subfigure}
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../stackoverflow.com/output/votesits/average_votes-i1.png}
|
|
\caption{An interrupted time series analysis of the vote score of questions created by new contributors on StackOverflow.com}
|
|
\label{stackoverflow_votesits}
|
|
\end{subfigure}\\
|
|
\begin{center}
|
|
\begin{subfigure}[c]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../stackoverflow.com/output/questionits/average_questions-i1.png}
|
|
\caption{An interrupted time series analysis of the number of questions created by new contributors on StackOverflow.com}
|
|
\label{stackoverflow_questionsits}
|
|
\end{subfigure}
|
|
\end{center}
|
|
\end{figure}
|
|
\pagebreak
|
|
% sentiment falling prior to change
|
|
% jump upward at the change
|
|
% sentiments rising after change
|
|
|
|
\section{AskUbuntu.com}
|
|
AskUbuntu sees a decrease in average sentiments prior to the change. After the introduction of the change, the regression dips but sentiments keep rising drastically since then. The vote score has a huge range of values prior to and after the change, however, the graph indicates the vote score declines after the change. The number of 1st questions slightly decreases prior to the change and starts rising after the change.
|
|
\begin{figure}[H]
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../askubuntu.com/output/its/average_sentiments-i1.png}
|
|
\caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on AskUbuntu.com}
|
|
\label{ubuntu_its}
|
|
\end{subfigure}
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../askubuntu.com/output/votesits/average_votes-i1.png}
|
|
\caption{An interrupted time series analysis of the vote score of questions created by new contributors on AskUbuntu.com}
|
|
\label{ubuntu_votesits}
|
|
\end{subfigure}\\
|
|
\begin{center}
|
|
\begin{subfigure}[c]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../askubuntu.com/output/questionits/average_questions-i1.png}
|
|
\caption{An interrupted time series analysis of the number of questions created by new contributors on AskUbuntu.com}
|
|
\label{ubuntu_questionsits}
|
|
\end{subfigure}
|
|
\end{center}
|
|
\end{figure}
|
|
\pagebreak
|
|
% senitments have gradually fallen prior to the change
|
|
% sentiments increased after the change
|
|
% maybe: sentiments did not change drastically as seen in maths communities
|
|
|
|
\section{ServerFault.com}
|
|
ServerFault shows gradually rising average sentiments prior to the change. At the time of the change, the regression makes a jump upward and the average sentiment decreases slowly afterward. The vote score falls prior to the change, made a huge jump upward, and quickly returns to the levels just prior to the change. The number of 1st questions, however, sees a drastic change. Prior to the change, the number of 1st questions decreases steadily, while after the change the numbers increase at the same pace as they fall prior to the change. The number of follow-up questions also sees the same course direction, falling prior and raising after the change.
|
|
\begin{figure}[H]
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../serverfault.com/output/its/average_sentiments-i1.png}
|
|
\caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on ServerFault.com}
|
|
\label{fault_its}
|
|
\end{subfigure}
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../serverfault.com/output/votesits/average_votes-i1.png}
|
|
\caption{An interrupted time series analysis of the vote score of questions created by new contributors on ServerFault.com}
|
|
\label{fault_votesits}
|
|
\end{subfigure}\\
|
|
\begin{center}
|
|
\begin{subfigure}[c]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../serverfault.com/output/questionits/average_questions-i1.png}
|
|
\caption{An interrupted time series analysis of the number of questions created by new contributors on ServerFault.com}
|
|
\label{fault_questionsits}
|
|
\end{subfigure}
|
|
\end{center}
|
|
\end{figure}
|
|
\pagebreak
|
|
% sentiments fairly stable before and after the change
|
|
% small jump in avg sentiments at change date
|
|
|
|
\section{stats.stackexchange.com}
|
|
On stats.stackexchange.com the average sentiment decreases steadily prior to the change. The regression dips when the change is introduced. However, the average sentiment after the change indicates a slight upward trend. The vote score also decreases prior to the change but does not recover afterward. However, the number of 1st questions and follow-up questions rise prior to the change and increase even faster after the change.
|
|
\begin{figure}[H]
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../stats.stackexchange.com/output/its/average_sentiments-i1.png}
|
|
\caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on stats.stackexchange.com}
|
|
\label{stats_its}
|
|
\end{subfigure}
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../stats.stackexchange.com/output/votesits/average_votes-i1.png}
|
|
\caption{An interrupted time series analysis of the vote score of questions created by new contributors on stats.stackexchange.com}
|
|
\label{stats_votesits}
|
|
\end{subfigure}\\
|
|
\begin{center}
|
|
\begin{subfigure}[c]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../stats.stackexchange.com/output/questionits/average_questions-i1.png}
|
|
\caption{An interrupted time series analysis of the number of questions created by new contributors on stats.stackexchange.com}
|
|
\label{stats_questionsits}
|
|
\end{subfigure}
|
|
\end{center}
|
|
\end{figure}
|
|
\pagebreak
|
|
% sentiments steadily decreasing prior to the change
|
|
% dip in avg sentiment at the change date
|
|
% sight upward trend after the change
|
|
|
|
\section{tex.stackexchange.com}
|
|
On tex.stackexchange.com the average sentiment is low compared to the other investigated data sets. Prior to the change the average sentiment only slightly decreases. When the change is introduced the regression takes a dip down and after the change, the average sentiment increases drastically. Future data will be required to see if this upward trend continues or evens out. In stark contrast, the vote score shows a downward trend, although there is a short window around the change date where vote scores are higher compared to before and after the change. The number of 1st questions has a downward trend before the change and an upward trend afterward. The downward trend of the number of follow-up questions is uninterrupted by the change.
|
|
\begin{figure}[H]
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../tex.stackexchange.com/output/its/average_sentiments-i1.png}
|
|
\caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on tex.stackexchange.com}
|
|
\label{tex_its}
|
|
\end{subfigure}
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../tex.stackexchange.com/output/votesits/average_votes-i1.png}
|
|
\caption{An interrupted time series analysis of the vote score of questions created by new contributors on tex.stackexchange.com}
|
|
\label{tex_votesits}
|
|
\end{subfigure}\\
|
|
\begin{center}
|
|
\begin{subfigure}[c]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../tex.stackexchange.com/output/questionits/average_questions-i1.png}
|
|
\caption{An interrupted time series analysis of the number of questions created by new contributors on tex.stackexchange.com}
|
|
\label{tex_questionsits}
|
|
\end{subfigure}
|
|
\end{center}
|
|
\end{figure}
|
|
\pagebreak
|
|
% avg sentiment fairly low compared to the other investigated communities
|
|
% avg sentiment slowly decreasing prior to the change
|
|
% large dips in avg snetiment after the change
|
|
% trend after change strongly upward
|
|
|
|
\section{unix.stackexchange.com}
|
|
On unix.stackexchange.com the average sentiment decreases prior to the change. When the change is introduced the regression takes a small dip down, however, the average sentiment increases fast after the change. The vote score shows a continuous downward trend and the number of 1st and follow-up questions fall slightly prior to the change and increase afterward.
|
|
\begin{figure}[H]
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../unix.stackexchange.com/output/its/average_sentiments-i1.png}
|
|
\caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on unix.stackexchange.com}
|
|
\label{unix_its}
|
|
\end{subfigure}
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../unix.stackexchange.com/output/votesits/average_votes-i1.png}
|
|
\caption{An interrupted time series analysis of the vote score of questions created by new contributors on unix.stackexchange.com}
|
|
\label{unix_votesits}
|
|
\end{subfigure}\\
|
|
\begin{center}
|
|
\begin{subfigure}[c]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../unix.stackexchange.com/output/questionits/average_questions-i1.png}
|
|
\caption{An interrupted time series analysis of the number of questions created by new contributors on unix.stackexchange.com}
|
|
\label{unix_questionsits}
|
|
\end{subfigure}
|
|
\end{center}
|
|
\end{figure}
|
|
\pagebreak
|
|
% sentiments decreasing prior to the change
|
|
% snetiments rising after the change
|
|
% little jump upwards at change date
|
|
|
|
|
|
|
|
% these communities befitted from the change
|
|
% #number of 1st questions rose in every of these communities
|
|
% #number of follow up questions are rising in most of the communities
|
|
% sentiment rose in most of the communities
|
|
% the vote score is mostly uncorrelated with the change
|
|
\section*{Benefitters}
|
|
More than half of the communities show benefits from the change. The number of first questions increase in all of the 6 previously shown communities. Also, for most of these communities the number of follow-up questions increased too. Furthermore, the sentiment ITS shows an improvement in all except 1 community. The vote score analysis yielded no meaningful results for these communities. The vote score does not change with the introduction of Stackexchange' policy, with the exception of ServerFault, however, the increase in the vote score did not last for long.
|
|
|
|
|
|
|
|
\section{math.stackexchange.com}
|
|
The math.stackexchange.com community shows a decrease in average sentiments, vote score, and the number of questions prior to the change. The measurements make a small jump upward when the change is introduced, however, they continue their downward trend after the introduction of the change. Only the number of follow-up questions stabilizes and begins to increase after the change.
|
|
\begin{figure}[H]
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../math.stackexchange.com/output/its/average_sentiments-i1.png}
|
|
\caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on math.stackexchange.com}
|
|
\label{math_its}
|
|
\end{subfigure}
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../math.stackexchange.com/output/votesits/average_votes-i1.png}
|
|
\caption{An interrupted time series analysis of the vote score of questions created by new contributors on math.stackexchange.com}
|
|
\label{math_votesits}
|
|
\end{subfigure}\\
|
|
\begin{center}
|
|
\begin{subfigure}[c]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../math.stackexchange.com/output/questionits/average_questions-i1.png}
|
|
\caption{An interrupted time series analysis of the number of questions created by new contributors on math.stackexchange.com}
|
|
\label{math_questionsits}
|
|
\end{subfigure}
|
|
\end{center}
|
|
\end{figure}
|
|
\pagebreak
|
|
% sentiments falling prior to the change
|
|
% sentiments falling faster than before the change
|
|
|
|
\section{MathOverflow.net}
|
|
MathOverflow shows a constant regression before the change, however, average sentiments are low at about 10 months before the change and spike high directly before the change. When the change is introduced the regression makes a small jump up and decreases thereafter. The votes score steadily increases prior to the change and then quickly returns to the level from 3 years before the change. The number of 1st questions falls prior to the change and stabilizes thereafter. This data set is sparse compared to the other datasets. Also, the vote scores are high compared to other datasets.
|
|
\begin{figure}[H]
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../mathoverflow.net/output/its/average_sentiments-i1.png}
|
|
\caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on MathOverflow.net}
|
|
\label{matho_its}
|
|
\end{subfigure}
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../mathoverflow.net/output/votesits/average_votes-i1.png}
|
|
\caption{An interrupted time series analysis of the vote score of questions created by new contributors on MathOverflow.net}
|
|
\label{matho_votesits}
|
|
\end{subfigure}\\
|
|
\begin{center}
|
|
\begin{subfigure}[c]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../mathoverflow.net/output/questionits/average_questions-i1.png}
|
|
\caption{An interrupted time series analysis of the number of questions created by new contributors on MathOverflow.net}
|
|
\label{matho_questionsits}
|
|
\end{subfigure}
|
|
\end{center}
|
|
\end{figure}
|
|
\pagebreak
|
|
% senitments stable/constant prior to the change
|
|
% falling after the change
|
|
|
|
\section{electronics.stackexchange.com}
|
|
On electronics.stackexchange.com the average sentiment and votes decrease continuously prior to the change. At the change date, the regression makes a little jump upward but the trend from before the change continues afterward. Similarly to SuperUser, the average sentiment recovers at about 12 months after the change is introduced and future data will be necessary to determine if the recovery is persistent. The number of 1st questions rises continuously prior to the change and decreases thereafter. The number of follow-up questions falls slightly prior to the change and stabilizes afterward.
|
|
\begin{figure}[H]
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../electronics.stackexchange.com/output/its/average_sentiments-i1.png}
|
|
\caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on electronics.stackexchange.com}
|
|
\label{ele_its}
|
|
\end{subfigure}
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../electronics.stackexchange.com/output/votesits/average_votes-i1.png}
|
|
\caption{An interrupted time series analysis of the vote score of questions created by new contributors on electronics.stackexchange.com}
|
|
\label{ele_votesits}
|
|
\end{subfigure}\\
|
|
\begin{center}
|
|
\begin{subfigure}[c]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../electronics.stackexchange.com/output/questionits/average_questions-i1.png}
|
|
\caption{An interrupted time series analysis of the number of questions created by new contributors on electronics.stackexchange.com}
|
|
\label{ele_questionsits}
|
|
\end{subfigure}
|
|
\end{center}
|
|
\end{figure}
|
|
\pagebreak
|
|
% sentiments were falling continuously before and after the change
|
|
% recovery started after 12 month after the change
|
|
% more data in the future will be required to determine if upward trend in the end continues
|
|
|
|
\section{SuperUser.com}
|
|
SuperUser shows only sightly decreasing average sentiment and vote score up to the change. At the change time the regressions take a dip down and the regression shows a downward trend after the change. Indeed the average sentiments and vote score dipped considerably when the change is introduced. The average sentiment recovers about 13 months later, while the vote score does not recover as well. The number of 1st questions decreases prior to the change and then goes through the roof indicating a huge wave of new users. This drastic influx of new users may explain the crash of the average sentiment and vote score that occurs at the same time. Data available in the future will show if the recovery is persistent.
|
|
\begin{figure}[H]
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../superuser.com/output/its/average_sentiments-i1.png}
|
|
\caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on SuperUser.com}
|
|
\label{super_its}
|
|
\end{subfigure}
|
|
\begin{subfigure}[t]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../superuser.com/output/votesits/average_votes-i1.png}
|
|
\caption{An interrupted time series analysis of the vote score of questions created by new contributors on SuperUser.com}
|
|
\label{super_votesits}
|
|
\end{subfigure}\\
|
|
\begin{center}
|
|
\begin{subfigure}[c]{0.5\textwidth}
|
|
\includegraphics[scale=0.37]{../superuser.com/output/questionits/average_questions-i1.png}
|
|
\caption{An interrupted time series analysis of the number of questions created by new contributors on SuperUser.com}
|
|
\label{super_questionsits}
|
|
\end{subfigure}
|
|
\end{center}
|
|
\end{figure}
|
|
\pagebreak
|
|
% sentiments fairly stable until the change date
|
|
% after change sentiments took a samll dive
|
|
% recovery after after 13 months to not quite the previous levels
|
|
|
|
\section*{No benefits/no evidence}
|
|
The 4 previously mentioned communities do not profit from the change. Although some communities improve in one statistic, they do not improve accross the field as shown in the other 6 communities. The 1st question statistic decreases in all 4 communities. With the exception of math.stackexchange.com, all of these communities do not improve in the followup question statistic. In all communities the vote score is on a (worse) downward trend after the change. Also, the sentiment values are decreasing after the change.
|
|
|
|
When looking at the results of SuperUser, the community stands out and shows interesting results. After about 6 mouths after the change the community the number of 1st questions triple. This level of new questions continues for 7 months before the the number go down towards the previous levels. In the same time frame the vote score and sentiment take a significant dive. After that the sentiment returns almost to the previous level while the vote score only increases mildly. However, this sudden increase in 1st questions and therefore users is not related to the change this thesis investigates.
|
|
|
|
%summary not working
|
|
% number of 1st questions does not increase after the change
|
|
% followup questions do improve in math.se, in others constant more or less
|
|
% vote scores are on a downward trend took
|
|
% sentiment scores started decreasing more rapid
|
|
% superuser oddball
|
|
|
|
|
|
\section*{Summary}
|
|
Summarizing, the change introduced by StackExchange clearly improved the engagement in 6 of the 10 investigated communities. Sentiment, vote score, and number (1st and follow-up questions) rose as a result. The other 4 communities do not profit from the change. Although, many statistics jump up to a higher level the downward trends are not stopped. The statistics of SuperUser show a large influx of new users about 6 months after the change sending the sentiment and vote score on a deep dive and with the decrease in new users they raise again. However, this event is not related to the change but the magnitude of the huge change in new user numbers renders the analysis uncomparable.
|