wip

text/3_method.tex

@@ -18,15 +18,11 @@ StackExchange introduced a \emph{new contributor} indicator to all communities o
 This thesis investigates the following criteria to determine whether the change affected a community positively or negatively, or whether the community is largly unaffected:
 \begin{itemize}
  \item \textbf{Sentiment of answers to a question}. This symbolizes the quality of communication between different individuals. Better values indicate better communication. Through the display of the \emph{new contributor} indicator, the answerer should react less negatively towards the new user when they behave outside the community standards.
- \item \textbf{Vote score of questions}. This is similar to the sentiment criterion. Voters will likely vote more postively (not voting instead of down voting, or upvoting instead of of not voting) due to the \emph{new contributor} indicator. Thereby the vote score should increase after the change.
- \item \textbf{The amount of first and follow-up question}. This symbolizes the willingness of users to participate in the community. Higher amounts of first questions indicate higher number of new participating users. Higher follow-up questions indicate that users are more willing to stay within the community.
+ \item \textbf{Vote score of questions}. This symbolizes the feedback the community give to a question. Voters will likely vote more postively (not voting instead of down voting, or upvoting instead of of not voting) due to the \emph{new contributor} indicator. Thereby the vote score should increase after the change.
+ \item \textbf{The amount of first and follow-up question}. This symbolizes the willingness of users to participate in the community. Higher amounts of first questions indicate higher number of new participating users. Higher follow-up questions indicate that users are more willing to stay within the community and continue their active participation.
 \end{itemize}
 If these criteria improve after the change is introducted, the community is affected positively. If they worsen, the community is affected negatively. If the criteria stay largely the same, then the community is unaffected. Here it is important to note that a question may receive answers and votes after the \emph{new contributor} indicator is no longer shown and therefore these are not considered as part of the data set to analyze.
 %only when new contributor insicator is shown
-%limitations
-% large sudden changes (maybe include example from analysis)
-% autocorrelation?
-% TODO
 
 
 
@@ -35,7 +31,7 @@ To measure the effect on sentiment of the change this thesis utilizes the Vader\
 However, just simply looking at the words in a text is not enough and therefore Vader also uses rules to determine how words are used in conjunction with other words. Some words can boost other words. For example, ``They did well.'' is less intense than ``They did extremely well.''. This works for both positive and negative sentences. Moreover, words can have different meanings depending on the context, for instance, ``Fire provides warmth.'' and ``Boss is about to fire an employee.'' This feature is called \emph{Word Sense Disambiguation}.
 
 Furthermore, Vader also detects language features commonly found in social media text which may not be present in other forms of text, for instance, books, or news papers. Social media texts may contain acronyms, initialisms (for instance \emph{afaik} (as far as I know)), slang words, emojis, caps words (often used to emphasize meaning), punctuation (for instance, \emph{!!!}, and \emph{?!?!}), etc.. These features can convey a lot of meaning and drastically change the sentiment of a text. 
-After all these features are considered, Vader outputs a sentiment value between -1 and 1 on a continuous range. The sentiment range is divided into 3 classes: negative (-1 to -0.05), neutral (-0.05 to 0.05), and positive (0.05 to 1). The outer edges of this range are rarely reached as the text would have to be extremely negative or positive which is very unlikely. 
+After all these features are considered, Vader assigns a sentiment value between -1 and 1 on a continuous range. The sentiment range is divided into 3 classes: negative (-1 to -0.05), neutral (-0.05 to 0.05), and positive (0.05 to 1). The outer edges of this range are rarely reached as the text would have to be extremely negative or positive which is very unlikely. 
 
 %speed
 Due to this mathematical simplicy, Vader is really fast when computing a sentiment value for a given text. This feature is one of the requirements   \citeauthor{hutto2014vader} originally posed. They proposed that Vader shall be fast enough to do online (real time) analysis of social media text.
@@ -79,6 +75,12 @@ After preprocessing the raw data, relevant data is filtered and computed. Questi
 An interrupted time series (ITS) analysis captures trends before and after a change in a system and fits very well with the question this thesis investigates. ITS can be applied to a large variety of data if the data contains the same kind of data points before and after the change and when the change date and time are known. \citeauthor{bernal2017interrupted} published a paper on how ITS works \cite{bernal2017interrupted}. ITS performes well on medical data, for instance, when a new treatment is introduced ITS can visualize if the treatment improves a condition. For ITS no control group is required and often control groups are not feasible. ITS only works with the before and after data and a point in time where a change was introduced. 
 ITS relies on linear regression and tries to fit a three-segment linear function to the data. The authors also described cases where more than three segments are used but these models quickly raise the complexity of the analysis and for this thesis a three-segment linear regression is sufficient. The three segments are lines to fit the data before and after the change as well as one line to connect the other two lines at the change date. Figure \ref{itsexample} shows an example of an ITS. Each segment is captured by a tensor of the following formula $Y_t = \beta_0 + \beta_1T + \beta_2X_t + \beta_3TX_t$, where $T$ represents time as a number, for instance, number of months since the start of data recording, $X_t$ represents 0 or 1 depending on whether the change is in effect, $\beta_0$ represents the value at $T = 0$, $\beta_1$ represents the slope before the change, $\beta_2$ represents the value when the change is introduced, and $\beta_3$ represents the slope after the change. Contrary to the basic method explained in \cite{bernal2017interrupted} where the ITS is performed on aggregated values per month, this thesis performs the ITS on single data points, as the premise that the aggregated values all have the same weight within a certain margin is not fulfilled for sentiment and vote score values. Performing the ITS with aggregated values would skew the linear regression more towards data points with less weight. Single data point fitting prevents this, as weight is taken into account with more data points. To filter out seasonal effects, the average value of all data points with the same month of all years is subtracted from the data points (i.e. subtract the average value of all Januaries from each data point in a January). This thesis uses the least squares method for regression.
 
+Although, the ITS analysis takes data density variability and seasonality into account, there is always a possibility that an unknown factor or event is contained in the data. It is always recommended to do a visual inspection of the data. This thesis contains one example where the data density increases so drastically in a particular time span that this form of analysis looses accuracy. 
+%limitations
+% large sudden changes (maybe include example from analysis)
+% autocorrelation?
+% 
+
 
 \begin{figure}
  \centering\includegraphics[scale=0.7]{figures/itsexample}