\chapter{Discussion}
%TODO ~1 ref/paragraph

The ITS analysis of the investigated communities shows mixed results. Some communities show an improvement in the measured qualities while others are not affected at all or show a decrease in these qualities. By and large, the majority of the investigated communities benefit from the change while the minority sees either no change or a change for the worse. Some communities show interesting features unrelated to the analysis which will also be mentioned in this chapter.

\section{Benefitters}
There are 6 communities that profit from the change in some form: StackOverflow, AskUbuntu, ServerFault, stats.stackexchange.com, tex.stackexchange.com, and unix.stackexchange.com. 

The StackOverflow community has a fairly stable average sentiment before the change. The average sentiment jumps to a higher level and keeps rising after the change is introduced. Furthermore, the number of 1st questions from new contributors starts rising drastically after the change while prior levels stagnate. Also, the follow-up questions start increasing slightly. The votes score trend takes a new direction 9 months before the change and is unrelated to it. The change has a positive effect on the StackOverflow community. The graph with the number of questions from new contributors shows a good example of seasonality in data \cite{bernal2017interrupted}. The month 0 indicates August. For the 1st questions, the months -44, -32, -20, -8, 4, and 16 are all a local minimum. These months are all in December. The months -31 to -27, -19 to -15, -7 to -3, and 5 to 9 show a pattern with 3 upward spikes. During December, the people of large portions of the world are going through a holiday season which may likely explain these regular dips in contribution. The graph for the follow-up questions also shows dips in the months of December.

AskUbuntu shows an interesting zig-zag pattern in the average sentiment graph every 20 months. However, this is not a seasonal effect, as seasonal effects are based on a 12-month cycle \cite{bernal2017interrupted}. Also, the average sentiment falls before the change and raises thereafter, indicating that the change works for this community. However, further data is needed to see if the zig-zag pattern repeats itself. The number of 1st questions starts increasing again after the change stopping the downward trend before that.

On stats.stackexchange.com the average sentiment falls before the change but since the change, the downward trend stops and the sentiment starts to rise slowly, suggesting the change has a positive effect on the community. This is supported by the increase in the number of 1st and follow-up questions by new contributors. The vote score takes a dip after the change but starts to recover after 12 months which could be the result of another factor. In the same time frame, the number of 1st questions increases a lot which means more new contributors contribute to the community. Due to this influx of new users, the community metrics suffer for a period of time but recover afterward. This effect is also described in \cite{lin2017better}. However, in this case the effect are not as pronounced.

In the tex.stackexchange.com community sentiments are stable before the change and show a stark rising pattern after the change. The change seems to work for this community but future data will be necessary to see if the rising pattern continues in the shown manner. The votes score ITS does not fit the model and values before and after the change indicate a linear downward trend. However, the number of 1st questions increases slightly after the change while the prior trend shows a decreasing development. The number of follow-up questions still continues a downward trend, indicating that the new contributors tend to become one-day-flies \cite{slag2015one}. By looking at the graph of the 1st questions, the months of -44, -32, -20, -8, 4, and 16 are local minima, indicating seasonality in the data \cite{bernal2017interrupted}. These months are all in December when the people of large parts of the world are on holiday.

unix.stackexchange.com also shows a decreasing pattern prior to and a rising pattern in sentiment after the change. The vote score analysis shows a fairly linear downward trend before and after the change and is not affected by it. However, the number of 1st questions by new contributors starts to drastically increase while before the change the levels are constant, indicating this community also profits from the change.  %TODO 1 ref, nothing found

On ServerFault the sentiment rises gradually before the change, jumps upward by a small value when the change is introduced and the sentiment falls slowly thereafter but the levels are pretty stable over the analyzed period. The vote scores show the change has a huge impact on the community. The previously decreasing trend jumps up by a large amount. However, the vote score rapidly returns to levels right before the change. Contrary, the number of first questions turns in direction and starts increasing at the same rate it is falling previously. %TODO 1 ref, nothing found

%~ - -
\section{No benefits/no evidence}
The other 4 communities do not seem to profit from the change so clearly: Mathoverflow, math.stackexchange.com, electronics.stackexchange.com, and SuperUser. Some of these communities still improve in certain aspects but the overall picture of the analysis does not allow an improving conclusion.

The average sentiment stays constant on MathOverflow before the change and decreases afterward. The sentiment levels start increasing six months before the change and are unrelated. However, the sentiment falls sharply at the change date, indicating the sentiment values are affected negatively by the change. The vote score is steadily increasing before the change and crashes shortly after the change. However, the vote score is very high compared to other communities. The number of 1st questions stabilizes after the change compared to the slight downward previously. By looking at the graph of the 1st questions, the months of -41, -29, -17, -5, and 7 are local maxima, indicating seasonality in the data \cite{bernal2017interrupted}. These months are all in March. Also while the number of 1st questions stabilizes to a constant trend, the number of follow-up questions decreases, indicating that the new users tend more to become one-day-flies as time passed on \cite{slag2015one}.

math.stackexchange.com shows a downward trend before and after the change in sentiment and vote score. The sentiment ITS is particularly affected by the low sentiment values at the end and future data is required to determine if this trend continues. However, the number of 1st questions stabilizes a bit after changes, and follow-up questions even see a slight increase after the change. The graph with the number of questions from new contributors shows a good example of seasonality in data \cite{bernal2017interrupted}. The month 0 indicates August. For the 1st questions, the months -44, -32, -20, -8, 4, and 16 are all a local minimum. These months are all in December. Similarly, the months -38 and -37, -26 and -25, -14 and -13, -2 and -1, and 10 and 11 are all in June and July. During both these times, the people large portions of the world are going through a holiday season which may likely explain these regular dips in contribution. The graph for the follow-up questions also shows dips at the same times, although the dips in December are not always as discernible.

The electronics.stackexchange.com community has a similar pattern for the sentiment value and vote scores compared to math.stackexchange.com. However, the sentiment values seem to recover after about 12 months and future data is required to see if the rise at the end of the period is a long-term trend. The rising number of first questions of new contributors stops at the change date and transitions into a decreasing pattern. However, the number of follow-up questions increases slightly after the change. Even though the number of new users decreases after the change the amount of follow-up questions increases, indicating the number of one-day-flies decreases \cite{slag2015one}.

SuperUser shows an odd pattern. The average sentiment values and votes scores are stable before the change and decrease dramatically shortly afterward. However, the sentiment recovers after 12 months. The ITS model chosen in this thesis is not able to capture the apparent pattern. However, the number of 1st questions skyrockets indicating a huge influx of new users. The time frames of the falling sentiment values and vote scores, and the rising number of first questions overlap, indicating the huge influx of new users is responsible for the falling patterns. This is a good example of the \emph{defaulting} described in \cite{lin2017better}. While the community metrics suffer for a period of time, they start to recover after some time. Also, while the number of 1st questions skyrockets, the number of follow-up questions stays largely the same, indicating most of the new users are in fact one-day-flies \cite{slag2015one}.

% similarities in results and differences
% so: only community that shows a clear improvement when comapred to prior to change sentiment
% mse not really changing, downward trend before and after, probably to the low last 5 values, its does not say much
% mo: sentiments largely the same before and after, regression falling due to how its works, its does not say much
% au: show intresting zigzag pattern: falling then gradually rising
% sf: sentiments fairly stable change not really detectable
% su: sneitments crashed down after change but recovered later, its model cannot catch this
% ele: steadily falling, same upward trend in the end, maybe senitments recover i future data
% stats: falling before change and drop at change but afterward seniemnts stabilized even rising 
% tex: sentiments took up a bit after the change; change seems to works
% unix: sentiments falling prior but gainig after; change seems to work

\section{Summary}
%TODO better headline
%DONE write about seasonality. math.stackexchange.com, MathOverflow, tex.stackexchange.com, AskUbuntu, StackOverflow
%DROP write about onedayflies. SuperUser, electronics.stackexchange.com, MathOverflow, tex.stackexchange.com, more likely dont write about this, justification too hard
%DONE write about defaulting
%TODO more refs?
By and large, the change introduced by the StackExchange team has a clear positive effect on more than half of the investigated communities. The sentiments of answers to questions of new contributors increase as well as the number of questions from new contributors. The vote score is not particularly affected by the change.

math.stackexchange.com is not really affected by the change, although the number of 1st questions stabilized a bit and follow-up questions from new contributors increase again. MathOverflow shows a similar picture. The sentiment on electronics.stackexchange.com also is not particularly affected by the change and continues to decrease.

Two of the communities, SuperUser and stats.stackexchange.com, have a delayed temporary decrease in sentiment which recovers after about 12 months, which may be attributable to the larger influx of new contributors \cite{lin2017better}. The selected ITS model is not designed to capture the sentiment pattern of these communities. 

Five of the communities, AskUbuntu, math.stackexchange.com, MathOverflow, StackOverflow, and tex.stackexchange.com, show seasonality \cite{bernal2017interrupted} in the number of contributions from new users. In most of these communities, in the month of December, the number of contributions falls to a local minimum. The most likely explanation is that large parts of the world are on holiday in the latter half of December, thus decreasing the number of contributions.

% expectations from before the experiment and how they match with results
% did change from SE produce the desired results?

Some investigated data sets show interesting patterns. StackOverflow shows the clearest results of all the investigated communities and closely resembles the example ITS shown in section 3. The result matches the expectation, that advising answerers to remember the code of conduct when answering questions from new contributors will increase the welcomingness and friendliness of the community, and shows that the change introduced by the StackExchange team works well for this community. 
%The AskUbuntu community shows an interesting zig-zag pattern where sentiment gradually rises over time and then falls abruptly.%TODO this sentence, wtf is it doing here, does not fit at all

% interesting single results?

The average sentiment of the StackOverflow community is the most stable in terms of deviation from the regression. This is expected as StackOverflow is the largest community by far and has the most questions created by newcomers. On the other hand, MathOverflow is the sparsest community and has the least amount of questions from new contributors. The level of the average sentiment also varies greatly between communities. stats.stackexchange.com has the highest level of average sentiment compared to the other communities, whereas, tex.stackexchange.com has the lowest level of average sentiment. MathOverflow has the highest level of vote scores by far. Also, in most communities, the number of questions from new contributors slowly decreases over time. This may be a result of the filling of gaps in the knowledge repository over time. %TODO ref for last sentence

% as expected #answers per month vary greatly -> mabye into data sets section
% some communties have a high average sentiment compared to others
% mathoverflow very sparse, stackoverflow densest obviously as biggets community 100x more users
% fewer questions by new users over time? + explaination

%future research %TODO may write something here
% investigate different change pattern and why they occured