I’m sharing summaries of Twitter numerical data from collecting the following bibliometrics event hashtags:
- #respbib18 (Responsible use of Bibliometrics in Practice, London, 30 January 2018) and
- #ResponsibleMetrics (The turning tide: A new culture of responsible metrics for research, London, 8 February 2018).
|Event title||Responsible use of Bibliometrics in Practice|
|Times||9:00 am – 4:30 pm||GMT|
|Number of links||128|
|Number of RTs||100|
|Number of Tweets||360|
|First Tweet in Archive||23/01/2018 11:44||GMT|
|Last Tweet in Archive||01/02/2018 16:17||GMT|
|In Reply Ids||15|
|In Reply @s||49|
|Unique users who used tag only once||26||<–for context of engagement|
|Event title||The turning tide: A new culture of responsible metrics for research|
|Times||09:30 – 16:00||GMT|
|Number of links||210|
|Number of RTs||318|
|Number of Tweets||796|
|First Tweet in Archive||05/02/2018 09:31||GMT|
|Last Tweet in Archive||08/02/2018 16:25||GMT|
|In Reply Ids||43|
|In Reply @s||76|
|Unique usernames who used tag only once||109||<–for context of engagement|
#respbib18: 30 Most Frequent Terms
#ResponsibleMetrics: 30 Most Frequent Terms
Twitter data mined with Tweepy. For robustness and quick charts a parallel collection was done with TAGS. Data was checked and deduplicated with OpenRefine. Text analysis performed with Voyant Tools. Text was anonymised through stoplists; two stoplists were applied (one to each dataset), including usernames and Twitter-specific terms (such as RT, t.co, HTTPS, etc.), including terms in hashtags. Event title keywords were not included in stoplists.
No sensitive, personal nor personally-identifiable data is contained in this data. Any usernames and names of individuals were removed at data refining stage and again from text analysis results if any remained.
Please note that both datasets span different number of days of activity, as indicated in the summary tables. Source data was refined but duplications might have remained, which would logically affect the resulting term raw frequencies, therefore numbers should be interpreted as indicative only and not as exact measurements. RTs count as Tweets and raw frequencies reflect the repetition of terms implicit in retweeting.
As usual I share this hoping others might find interesting and draw their own conclusions.
A very general insight for me is that we need a wider group engaging with this discussions. At most we are talking about a group of approximately 50 individuals that actively engaged on Twitter on both events.
From the Activity charts it is noticeable that tweeting recedes at breakout times, possibly indicating that most tweeting activity is coming from within the room– when hashtags create wide engagement, activity is more constant and does not exactly reflect the timings of actual real-time activity in the room.
It seems to me that the production, requirement, use and interpretation of metrics for research assessment directly affects everyone in higher education, regardless of their position or role. The topic should not be obscure or limited to bibliometricians and RDM, Research and Enterprise or REF panel people.
Needless to say I do not think everyone ‘engaged’ with these events or topics is or should be actively using the hashtag on Twitter (i.e. we don’t know how many people followed on Twitter). An assumption here is that we cannot detect nor measure anything if there is not a signal– more folks elsewhere might be interested in these events but if they did not use the hashtag they were logically not detected here. That there is no signal measurable with the selected tools does not mean there is not a signal elsewhere, and I’d like this to be a comment on metrics for assessment as well.
In terms of frequent terms it remains apparent (as in other text analyses I have performed on academic Twitter hashtag archives) that frequently tweeted terms remain ‘neutral’ nouns, or adjectives if they are a keyword in the event’s title, subtitle or panel sessions (e.g. ‘responsible’). When a term like ‘snowball’ or ‘bonkers’ appears, it stands out. Due to the lack of more frequent modifiers, it remains hard to distant-read sentiment or critical stances, or even positions. Most frequent terms do come from RTs, not because of consensus in ‘original’ Tweets.
It seems that if we wanted to demonstrate the value added by live-tweeting or using an event’s hashtag remotely, quantifying (metricating?) the active users, tweets over time, days of activity and frequent words would not be the way to go for all events, particularly not for events with relatively low Twitter activity.
As we have seen, automated text analysis is more likely to reveal mostly-neutral keywords, rather than any divergence of opinion on or additions to the official discourse. We would have to look at those words less repeated, and perhaps to replies that did not use the hashtag, but this is not recommended as it would complicate things ethically: though it is generally accepted that RTs do not imply endorsement, less frequent terms in Tweets with the hashtag could single-out individuals, and if a hashtag was not included on a Tweet it should be interpreted the Tweet is not meant to be part of that public discussion/corpus.