wea_ondara/master
1
1
Fork 0
Code Issues Pull Requests Releases Wiki Activity

wip

Browse Source
This commit is contained in:
wea_ondara
2020-05-02 11:52:30 +02:00
parent a02705666e
commit 9d195f0f68
7 changed files with 52 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
text/3_method.tex
View File

@@ -14,7 +14,7 @@ StackExchange introduced a \emph{new contributor} indicator to all communities o
% new user indicator visible for 1 week ... % new user indicator visible for 1 week ...
%TODO more vader explanation %TODO more vader explanation
To measure the effectiveness of the change we chose Vader, a sentiment analysis tool designed for social media interactions \cite{hutto2014vader}. Vader uses a lexicon of words with attached sentiment values and rules related to grammar and syntax to determine a sentiment value between -1 and 1 to a given piece of text. The sentiment range is divided into 3 classes: negative (-1 to -0.05), neutral (-0.05 to 0.05), and positive (0.05 to 1). The outer edges of the value space are rarely reached as the text would have to be negative or positive to the extremes which is very unlikely. To measure the effectiveness of the change this thesis utilizes Vader, a sentiment analysis tool with exceptional performance in analysing and categorizing microblog-like texts as well as good generalization in other domains \cite{hutto2014vader}. The choice is based on the speed and simplicity of Vader. Vader uses a lexicon of words with attached sentiment values and rules related to grammar and syntax to determine a sentiment value between -1 and 1 to a given piece of text. The sentiment range is divided into 3 classes: negative (-1 to -0.05), neutral (-0.05 to 0.05), and positive (0.05 to 1). The outer edges of the value space are rarely reached as the text would have to be extremely negative or positive which is very unlikely. This design allows fast and verifiable analysis.
% sentiment calculation via vaderlib, write whole paragraph and explain, also add ref to paper \cite{hutto2014vader} % sentiment calculation via vaderlib, write whole paragraph and explain, also add ref to paper \cite{hutto2014vader}
@@ -44,6 +44,14 @@ After preprocessing the raw data, relevant data is filtered and computed. Questi
\section{Analysis} \section{Analysis}
An interrupted time series (ITS) analysis captures trends before and after a change in a system and fits very well with the question this thesis investigates. ITS can be applied to a large variety of data if the data contains the same kind of data points before and after the change and when the change date and time are known. \citeauthor{bernal2017interrupted} published a paper on how ITS works \cite{bernal2017interrupted}. ITS works well on medical data, for instance, when a new treatment is introduced ITS can visualize if the treatment improves a condition. For ITS no control group is required and often control groups are not feasible. ITS only works with the before and after data and a date where a change was introduced. An interrupted time series (ITS) analysis captures trends before and after a change in a system and fits very well with the question this thesis investigates. ITS can be applied to a large variety of data if the data contains the same kind of data points before and after the change and when the change date and time are known. \citeauthor{bernal2017interrupted} published a paper on how ITS works \cite{bernal2017interrupted}. ITS works well on medical data, for instance, when a new treatment is introduced ITS can visualize if the treatment improves a condition. For ITS no control group is required and often control groups are not feasible. ITS only works with the before and after data and a date where a change was introduced.
ITS relies on linear regression and tries to fit a three-segment linear function to the data. The authors also described cases where more than three segments are used but these models quickly raise the complexity of the analysis and for this thesis a three-segment linear regression is sufficient. The three segments are lines to fit the data before and after the change as well as one line to connect the other two lines at the change date. Figure \ref{itsexample} shows an example of an ITS. Each segment is captured by a tensor of the following formula $Y_t = \beta_0 + \beta_1T + \beta_2X_t + \beta_3TX_t$, where $T$ represents time as a number, for instance, number of months since the start of data recording, $X_t$ represents 0 or 1 depending on whether the change is in effect, $\beta_0$ represents the value at $T = 0$, $\beta_1$ represents the slope before the change, $\beta_2$ represents the value when the change is introduced, and $\beta_3$ represents the slope after the change. Contrary to the method in \cite{bernal2017interrupted} where the ITS is performed on aggregated values per month, this thesis performs the ITS on single data points, as the premise that the aggregated values all have the same weight within a certain margin is not fulfilled. Performing the ITS with aggregated values would skew the linear regression more towards data points with less weight. Single data point fitting prevents this, as weight is taken into account with more data points. ITS relies on linear regression and tries to fit a three-segment linear function to the data. The authors also described cases where more than three segments are used but these models quickly raise the complexity of the analysis and for this thesis a three-segment linear regression is sufficient. The three segments are lines to fit the data before and after the change as well as one line to connect the other two lines at the change date. Figure \ref{itsexample} shows an example of an ITS. Each segment is captured by a tensor of the following formula $Y_t = \beta_0 + \beta_1T + \beta_2X_t + \beta_3TX_t$, where $T$ represents time as a number, for instance, number of months since the start of data recording, $X_t$ represents 0 or 1 depending on whether the change is in effect, $\beta_0$ represents the value at $T = 0$, $\beta_1$ represents the slope before the change, $\beta_2$ represents the value when the change is introduced, and $\beta_3$ represents the slope after the change. Contrary to the method in \cite{bernal2017interrupted} where the ITS is performed on aggregated values per month, this thesis performs the ITS on single data points, as the premise that the aggregated values all have the same weight within a certain margin is not fulfilled. Performing the ITS with aggregated values would skew the linear regression more towards data points with less weight. Single data point fitting prevents this, as weight is taken into account with more data points.
%TODO include ITS example img
\begin{figure}
\centering\includegraphics[scale=0.7]{figures/itsexample}
\caption{An example that visualizes how ITS works. The change of the system occurs at month 0. The blue line shows the average sentiment of fictional answers grouped by month. The numbers attached to the blue line show the number of sentiment values for a given month. The yellow line represents the ITS analysis as a three-segment line.}
\label{itsexample}
\end{figure}
%interrupted time series %interrupted time series
% ref tutorial paper \cite{bernal2017interrupted} % ref tutorial paper \cite{bernal2017interrupted}

31
text/4_datasets.tex
View File

@@ -1,7 +1,9 @@
\chapter{Datasets} \chapter{Datasets}
%TODO maybe more text
%general %general
StackExchange provides complete datasets of its communities for research purposes on archive.org \cite{archivestackexchange}. This thesis investigates the largest dataset available and include dataset for following communities: StackExchange provides complete datasets of its communities for research purposes on archive.org \cite{archivestackexchange}. StackExchange also provides a short guide on how to interpret the provided data, as some data values are strictly numerical and do not convey any meaning without the knowledge of what these values represent.
This thesis investigates the largest datasets available and includes the datasets of the following communities:
\begin{itemize} \begin{itemize}
\item StackOverflow.com \item StackOverflow.com
\item math.stackexchange.com \item math.stackexchange.com
@@ -14,20 +16,19 @@ StackExchange provides complete datasets of its communities for research purpose
\item tex.stackexchange.com \item tex.stackexchange.com
\item unix.stackexchange.com \item unix.stackexchange.com
\end{itemize} \end{itemize}
These datasets are selected due to their size as bigger datasets yield more consistent results. Smaller datasets may be too sparse to take any meaningful conclusions. Also, outliers would influence the results more compared to bigger datasets. These datasets are selected due to their size as larger datasets yield more consistent results. Smaller datasets may be too sparse to take any meaningful conclusions. Also, outliers would influence the results more when compared to outliner in bigger datasets. The dataset contain all the necessary data since the creation of the respective community and until the last day of February 2020.
% from archive.org \cite{archivestackexchange} % from archive.org \cite{archivestackexchange}
% list of datasets % list of datasets
% selected largest dataset, smaller datasets data to sparse to take concolusions, statistcal change of outliner to big, outlines would effect the outcome by too much % selected largest dataset, smaller datasets data to sparse to take concolusions, statistcal change of outliner to big, outlines would effect the outcome by too much
% larger data sets yield more consistent results % larger data sets yield more consistent results
% dataset include data since inception of community until some date
%dataset include data since inception of community until some date %TODO find last dates
%sections 1 per site %sections 1 per site
\section{StackOverflow.com} %TODO insert values \section{StackOverflow.com} datavalues not computed yet. %TODO insert values
StackOverflow is the largest and oldest community of the StackExchange platform. StackOverflow is the largest and oldest community of the StackExchange platform.
The community has 165567 registered users of which 3467 were active in May of 2019. The community has 165567 registered users of which 3467 were active between December 2019 and February 2020.
Members asked 116797 questions in total and gave 202751 answers with an average answer density of 1.73 answers per question. Members asked 116797 questions in total and gave 202751 answers with an average answer density of 1.73 answers per question.
New users asked 42996 questions with an average of 1.129 questions per new user during their first week after registration. New users asked 42996 questions with an average of 1.129 questions per new user during their first week after registration.
@@ -46,7 +47,7 @@ New users asked 42996 questions with an average of 1.129 questions per new user
\section{math.stackexchange.com} \section{math.stackexchange.com}
``Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields.'' \cite{mathstackexchangecom} ``Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields.'' \cite{mathstackexchangecom}
The community has 624671 registered users of which 17074 were active in February of 2020. The community has 624671 registered users of which 17074 were active between December 2019 and February 2020.
Members asked 1170938 questions in total and gave 1565188 answers with an average answer density of 1.336 answers per question. Members asked 1170938 questions in total and gave 1565188 answers with an average answer density of 1.336 answers per question.
New users asked 265704 questions with an average of 1.336 questions per new user during their first week after registration. New users asked 265704 questions with an average of 1.336 questions per new user during their first week after registration.
@@ -65,7 +66,7 @@ New users asked 265704 questions with an average of 1.336 questions per new user
\section{MathOverflow.net} \section{MathOverflow.net}
MathOverflow.net is a rather small community for professional mathematicians. MathOverflow.net is a rather small community for professional mathematicians.
The community has 105471 registered users of which 1501 were active in February of 2020. The community has 105471 registered users of which 1501 were active between December 2019 and February 2020.
Members asked 108083 questions in total and gave 144918 answers with an average answer density of 1.34 answers per question. Members asked 108083 questions in total and gave 144918 answers with an average answer density of 1.34 answers per question.
New users asked 23746 questions with an average of 1.131 questions per new user during their first week after registration. New users asked 23746 questions with an average of 1.131 questions per new user during their first week after registration.
@@ -84,7 +85,7 @@ New users asked 23746 questions with an average of 1.131 questions per new user
\section{AskUbuntu.com} \section{AskUbuntu.com}
AskUbuntu.com is a rather small community for Ubuntu users and developers. AskUbuntu.com is a rather small community for Ubuntu users and developers.
The community has 783614 registered users of which 7033 were active in February of 2020. The community has 783614 registered users of which 7033 were active between December 2019 and February 2020.
Members asked 334194 questions in total and gave 418051 answers with an average answer density of 1.25 answers per question. Members asked 334194 questions in total and gave 418051 answers with an average answer density of 1.25 answers per question.
New users asked 157018 questions with an average of 1.101 questions per new user during their first week after registration. New users asked 157018 questions with an average of 1.101 questions per new user during their first week after registration.
@@ -103,7 +104,7 @@ New users asked 157018 questions with an average of 1.101 questions per new user
\section{ServerFault.com} \section{ServerFault.com}
ServerFault.com is a rather small community for system and network administrators. ServerFault.com is a rather small community for system and network administrators.
The community has 451180 registered users of which 3947 were active in February of 2020. The community has 451180 registered users of which 3947 were active between December 2019 and February 2020.
Members asked 274564 questions in total and gave 432334 answers with an average answer density of 1.574 answers per question. Members asked 274564 questions in total and gave 432334 answers with an average answer density of 1.574 answers per question.
New users asked 88547 questions with an average of 1.106 questions per new user during their first week after registration. New users asked 88547 questions with an average of 1.106 questions per new user during their first week after registration.
@@ -122,7 +123,7 @@ New users asked 88547 questions with an average of 1.106 questions per new user
\section{SuperUser.com} \section{SuperUser.com}
SuperUser.com is a rather small community for computer enthusiasts and power users. SuperUser.com is a rather small community for computer enthusiasts and power users.
The community has 861533 registered users of which 7392 were active in February of 2020. The community has 861533 registered users of which 7392 were active between December 2019 and February 2020.
Members asked 424718 questions in total and gave 587559 answers with an average answer density of 1.383 answers per question. Members asked 424718 questions in total and gave 587559 answers with an average answer density of 1.383 answers per question.
New users asked 161397 questions with an average of 1.085 questions per new user during their first week after registration. New users asked 161397 questions with an average of 1.085 questions per new user during their first week after registration.
@@ -141,7 +142,7 @@ New users asked 161397 questions with an average of 1.085 questions per new user
\section{electronics.stackexchange.com} \section{electronics.stackexchange.com}
electronics.stackexchange.com is a rather small community for electrical engineering. electronics.stackexchange.com is a rather small community for electrical engineering.
The community has 184795 registered users of which 3172 were active in February of 2020. The community has 184795 registered users of which 3172 were active between December 2019 and February 2020.
Members asked 130025 questions in total and gave 221811 answers with an average answer density of 1.705 answers per question. Members asked 130025 questions in total and gave 221811 answers with an average answer density of 1.705 answers per question.
New users asked 47035 questions with an average of 1.126 questions per new user during their first week after registration. New users asked 47035 questions with an average of 1.126 questions per new user during their first week after registration.
@@ -160,7 +161,7 @@ New users asked 47035 questions with an average of 1.126 questions per new user
\section{stats.stackexchange.com (Cross Validated)} \section{stats.stackexchange.com (Cross Validated)}
``Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization.'' \cite{statsstackexchangecom} ``Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization.'' \cite{statsstackexchangecom}
The community has 227032 registered users of which 4485 were active in February of 2020. The community has 227032 registered users of which 4485 were active between December 2019 and February 2020.
Members asked 151777 questions in total and gave 148046 answers with an average answer density of 0.975 answers per question. Members asked 151777 questions in total and gave 148046 answers with an average answer density of 0.975 answers per question.
New users asked 57636 questions with an average of 1.112 questions per new user during their first week after registration. New users asked 57636 questions with an average of 1.112 questions per new user during their first week after registration.
@@ -179,7 +180,7 @@ New users asked 57636 questions with an average of 1.112 questions per new user
\section{tex.stackexchange.com} \section{tex.stackexchange.com}
tex.stackexchange.com is a rather small community for TEX and related typesetting systems. tex.stackexchange.com is a rather small community for TEX and related typesetting systems.
The community has 171867 registered users of which 3280 were active in February of 2020. The community has 171867 registered users of which 3280 were active between December 2019 and February 2020.
Members asked 188860 questions in total and gave 227875 answers with an average answer density of 1.206 answers per question. Members asked 188860 questions in total and gave 227875 answers with an average answer density of 1.206 answers per question.
New users asked 59692 questions with an average of 1.191 questions per new user during their first week after registration. New users asked 59692 questions with an average of 1.191 questions per new user during their first week after registration.
@@ -198,7 +199,7 @@ New users asked 59692 questions with an average of 1.191 questions per new user
\section{unix.stackexchange.com} \section{unix.stackexchange.com}
unix.stackexchange.com is a rather small community for Linux and Unix-like operating systems. unix.stackexchange.com is a rather small community for Linux and Unix-like operating systems.
The community has 356498 registered users of which 4565 were active in February of 2020. The community has 356498 registered users of which 4565 were active between December 2019 and February 2020.
Members asked 174625 questions in total and gave 256007 answers with an average answer density of 1.466 answers per question. Members asked 174625 questions in total and gave 256007 answers with an average answer density of 1.466 answers per question.
New users asked 62437 questions with an average of 1.124 questions per new user during their first week after registration. New users asked 62437 questions with an average of 1.124 questions per new user during their first week after registration.

25
text/5_results.tex
View File

@@ -1,51 +1,56 @@
\chapter{Results} \chapter{Results}
%TODO some text here
%TODO write some text to each result
\section{StackOverflow.com} \section{StackOverflow.com}
\begin{figure}[H] \begin{figure}[H]
% \centering\includegraphics[scale=0.47]{../stackoverflow.com/output/its/average_sentiments-i1.png} % \centering\includegraphics[scale=0.47]{../stackoverflow.com/output/its/average_sentiments-i1.png}
\caption{An interrupted time series analysis of the sentiments of answer to questions be new contributors on StackOverflow.com} \caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on StackOverflow.com}
\label{stackoverflow_its} \label{stackoverflow_its}
\end{figure} \end{figure}
\section{math.stackexchange.com} \section{math.stackexchange.com}
\begin{figure}[H] \begin{figure}[H]
\centering\includegraphics[scale=0.47]{../math.stackexchange.com/output/its/average_sentiments-i1.png} \centering\includegraphics[scale=0.47]{../math.stackexchange.com/output/its/average_sentiments-i1.png}
\caption{An interrupted time series analysis of the sentiments of answer to questions be new contributors on math.stackexchange.com} \caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on math.stackexchange.com}
\label{math_its} \label{math_its}
\end{figure} \end{figure}
\section{MathOverflow.net} \section{MathOverflow.net}
\begin{figure}[H] \begin{figure}[H]
\centering\includegraphics[scale=0.47]{../mathoverflow.net/output/its/average_sentiments-i1.png} \centering\includegraphics[scale=0.47]{../mathoverflow.net/output/its/average_sentiments-i1.png}
\caption{An interrupted time series analysis of the sentiments of answer to questions be new contributors on MathOverflow.com} \caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on MathOverflow.com}
\label{matho_its} \label{matho_its}
\end{figure} \end{figure}
\section{AskUbuntu.com} \section{AskUbuntu.com}
\begin{figure}[H] \begin{figure}[H]
\centering\includegraphics[scale=0.47]{../askubuntu.com/output/its/average_sentiments-i1.png} \centering\includegraphics[scale=0.47]{../askubuntu.com/output/its/average_sentiments-i1.png}
\caption{An interrupted time series analysis of the sentiments of answer to questions be new contributors on AskUbuntu.com} \caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on AskUbuntu.com}
\label{ubuntu_its} \label{ubuntu_its}
\end{figure} \end{figure}
\section{ServerFault.com} \section{ServerFault.com}
\begin{figure}[H] \begin{figure}[H]
\centering\includegraphics[scale=0.47]{../serverfault.com/output/its/average_sentiments-i1.png} \centering\includegraphics[scale=0.47]{../serverfault.com/output/its/average_sentiments-i1.png}
\caption{An interrupted time series analysis of the sentiments of answer to questions be new contributors on ServerFault.com} \caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on ServerFault.com}
\label{fault_its} \label{fault_its}
\end{figure} \end{figure}
\section{SuperUser.com} \section{SuperUser.com}
\begin{figure}[H] \begin{figure}[H]
\centering\includegraphics[scale=0.47]{../superuser.com/output/its/average_sentiments-i1.png} \centering\includegraphics[scale=0.47]{../superuser.com/output/its/average_sentiments-i1.png}
\caption{An interrupted time series analysis of the sentiments of answer to questions be new contributors on SuperUser.com} \caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on SuperUser.com}
\label{super_its} \label{super_its}
\end{figure} \end{figure}
\section{electronics.stackexchange.com} \section{electronics.stackexchange.com}
\begin{figure}[H] \begin{figure}[H]
\centering\includegraphics[scale=0.47]{../electronics.stackexchange.com/output/its/average_sentiments-i1.png} \centering\includegraphics[scale=0.47]{../electronics.stackexchange.com/output/its/average_sentiments-i1.png}
\caption{An interrupted time series analysis of the sentiments of answer to questions be new contributors on electronics.stackexchange.com} \caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on electronics.stackexchange.com}
\label{ele_its} \label{ele_its}
\end{figure} \end{figure}
% pvalues ... % pvalues ...
@@ -53,20 +58,20 @@
\section{stats.stackexchange.com} \section{stats.stackexchange.com}
\begin{figure}[H] \begin{figure}[H]
\centering\includegraphics[scale=0.47]{../stats.stackexchange.com/output/its/average_sentiments-i1.png} \centering\includegraphics[scale=0.47]{../stats.stackexchange.com/output/its/average_sentiments-i1.png}
\caption{An interrupted time series analysis of the sentiments of answer to questions be new contributors on stats.stackexchange.com} \caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on stats.stackexchange.com}
\label{stats_its} \label{stats_its}
\end{figure} \end{figure}
\section{tex.stackexchange.com} \section{tex.stackexchange.com}
\begin{figure}[H] \begin{figure}[H]
\centering\includegraphics[scale=0.47]{../tex.stackexchange.com/output/its/average_sentiments-i1.png} \centering\includegraphics[scale=0.47]{../tex.stackexchange.com/output/its/average_sentiments-i1.png}
\caption{An interrupted time series analysis of the sentiments of answer to questions be new contributors on tex.stackexchange.com} \caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on tex.stackexchange.com}
\label{tex_its} \label{tex_its}
\end{figure} \end{figure}
\section{unix.stackexchange.com} \section{unix.stackexchange.com}
\begin{figure}[H] \begin{figure}[H]
\centering\includegraphics[scale=0.47]{../unix.stackexchange.com/output/its/average_sentiments-i1.png} \centering\includegraphics[scale=0.47]{../unix.stackexchange.com/output/its/average_sentiments-i1.png}
\caption{An interrupted time series analysis of the sentiments of answer to questions be new contributors on unix.stackexchange.com} \caption{An interrupted time series analysis of the sentiments of answer to questions created by new contributors on unix.stackexchange.com}
\label{unix_its} \label{unix_its}
\end{figure} \end{figure}

8
text/6_discussion.tex
View File

@@ -1 +1,9 @@
\chapter{Discussion} \chapter{Discussion}
% similarities in results and differences
% expectations from before the experiment and how they match with results
% did change from SE produce the desired results?
%future research
% investigate different change pattern and why they occured

4
text/7_conclusion.tex
View File

@@ -1 +1,5 @@
\chapter{Conclusion} \chapter{Conclusion}
%sum up findings
% change did something?
%...

BIN
text/figures/itsexample.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 66 KiB

BIN
text/figures/new_contributor.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 28 KiB