Använd Google Analytics för att utvärdera och utveckla dina kurser

Kap 12 av P. Clint Rogers, Mary R. McEwen och SaraJoy Pond i Veletsianos bok Emerging Technologies in Distance Education (2010), handlar om att vi bör bli bättre på att utnyttja användarstatistiken som kan samlas in kring hur eleverna använder ett kursmaterial eller webben i stort. På så vis skulle vi bli bättre på att utvärdera och utveckla våra material och arbetsmetoder:

"Simply because students are at a distance, educators do not get the same kinds of immediate explicit and implicit feedback that comes when face to face. Web analytics provide an incredible opportunity for educators to receive helpful information regarding their students’ usage and behaviour patterns — on a scale that has the potential to transform the entire industry. Utilized to date primarily in business to track the online behaviour of consumer groups and to test related marketing efforts, web analytics can also be used in distance education to improve the tracking of learner behaviour and to test the impact made by changes in content or presentation." (Veletsiano, s 231).

Detta kapitel tyckte jag var mest intressant i den tredje delen av Veletianos bok och jag gillade särskilt den här jämförelsen där artikelförfattarna jämför den statistik man kan välja att plocka fram kring studenternas webbanvändning med vilka data som man valt att visa på en bils instrumentpanel:

"What are the main objectives that you want your website to accomplish? Which metrics will provide you the most meaningful information about how well you are accomplishing them? Take the dashboard of your car as an analogy. Many metrics could be tracked and reported about the current state of your car: combustion chamber temperature, fuel/oxygen mixture, fan belt RPM, and many more. But only a few are displayed on your dashboard, and most of us actively use only a selection of those. So, why are fuel level, speed, and turn signal functionality displayed on your dashboard while coolant level and spark plug efficiency are not? First, because these former metrics have direct, observable consequences to you, the operator (we all know how costly speeding tickets can be). Second, because you, the operator, can do something about them (the pressure of your foot on the accelerator directly affects your speed). Similarly, certain web analytics metrics become KPIs because of their impact on the ultimate outcome, as well as your ability to make actionable decisions based on them." (Veletsiano, s 233).

Författarna tar också upp ett väldigt passande citat från Albert Einstein:

“Not everything that can be measured is important, and not everything that is important can be measured.”

Jag använder mig mycket av bloggar i min undervisning och eftersom bloggar är ganska lätta att studera användarstatistiken för samtidigt som det blir så mycket data att det är lätt att känna sig vilse, så tänker jag att jag skulle vilja bli mer medveten om vad jag egentligen vill veta och hur jag ska ta reda på det. I studien använde man sig av Google Analytics och det gjorde mig ännu mer intresserad, eftersom det är det verktyg som jag själv använder.
I artikeln redovisas en studie som gav följande resultat med hjälp av statistiken:

Några slutsatser man drog utifrån studenternas användarstatistik var att eleverna tog del av materialet för i genomsnitt en lektion per tillfälle som de studerade:

"The analysts noticed a simple yet interesting correlation between page views per visit and pages per lesson. The averages of each of these metrics were quite close: average page views per visit = 6.5, and average pages per lesson = 5.09, suggesting that on average, a student’s plan is to complete one lesson in a sitting. This is something to consider when determining the amount of content one lesson should cover." (Veletsiano, s 238).

Man fann också, föga förvånande, att de första sidan på varje avsnitt fick särskilt många besökare, medan antalet läsare minskade längre fram i materialet:

"The first page of each lesson had many more unique page views than any other page in the lesson. The unique page views fell sharply even between the first and second page of each lesson. Then they remained fairly constant, until the last page of the lesson where there was a very noticeable uptick in unique page views. (Veletsiano, s 240).

Något som man ville ta reda på var om eleverna utnyttjade möjligheten att de hade tillgång till materialet 24 timmar om dygnet, 7 dagar i veckan, och det visade sig att de flesta jobbade med mellan kl 10 och 22 måndagar till och med torsdagar och att de flesta slutade tidigt på fredagarna och vilade under helgen:

"If “anytime” and “anywhere” are important KPIs, then there should be some supportive evidence of that in the analytics data. The client was quite interested in the “where” and “when” data summary. Site access was reported by time of day and by day of week. Not surprisingly, the biggest day for site access was Monday, with Tuesday through Thursday access being fairly equal, and then falling off on Friday, with Saturday and Sunday being the days with the least traffic. As far as time of day: timing was split fairly evenly between 10 AM and 10 PM, with the hours of noon to 4 PM slightly heavier. It is important to note that often the metrics of initial interest to a client are not necessarily the ones that can yield the most relevant insights or have the most impact on the outcomes." (Veletiano, s 242f).

Avslutningsvis gav artikelförfattarna några generella råd till den som vill använda sig av användarstatistiken (Veletsiano, s 244):

• "First, we must define the goals/objectives of the interaction. /.../
• Secondly, we must measure both the outputs and outcomes of the interaction. /.../
• Thirdly (perhaps most the crucial objective), we must use the resulting data to make improvements in the interaction. A question to be asked is this: “What would I do with this information if I had it?” If you do not know how you are going to use the data and what changes you will make as a result of different possible outcomes, then you should consider exploring other metrics where there is some actionable outcome. /.../ 
• Finally, analytics data thus interpreted and utilized may be shared to the benefit of users, other practitioners, and the distance education community as a whole."

De gav också lite vägledning kring vad man behöver tänka mer kring inför framtiden:

"The potential to collect and analyze real-time data from vast numbers of students could teach us a lot about how people interact with and learn in online learning environments. In reaching this potential, certain questions deserve more exploration:

• How do different segments of students (geography, age, gender, education level, major, etc.) interact with online resources? What are the common KPIs applicable to industry? 

• What are the most effective ways to implement multivariate testing? 

• How can web analytics be used to give individual students information regarding the relation of their own use patterns and results of others with similar patterns? 

• What are the best ways to automate some decisions based upon data indices? 

• How can the data gained through web analytics be combined with other evaluation methods (e.g., qualitative methods) in order to give a more complete picture of learner intent? 

• How will data-driven decision-making through the use of web analytics change the processes by which distance education is designed and evaluated?" (Veletsiano, s 245f).