WEBVTT

1
00:00:00.001 --> 00:00:09.000
Okay. Welcome to the session on integrating survey data from different sources. We have a mix.

2
00:00:09.000 --> 00:00:18.000
Of different modes. We have a protocol with presentation with which we will start. We have 2 remote speakers and 2 on-site speakers.

3
00:00:18.000 --> 00:00:30.000
So basically we decided to We happen to have all the possibilities covered. And we will get, you will get 15 min for your presentation.

4
00:00:30.000 --> 00:00:41.000
I will be. Timing it more or less accurately. And we will have the discussion section at the end for all presentations together.

5
00:00:41.000 --> 00:00:54.000
So that's it in terms of organization and I think we can always start with Yeah. I don't know why that's He actually meant to stop.

6
00:00:54.000 --> 00:01:07.000
Okay, so we have a message of acknowledgment which you may already have seen in the previous session we want to acknowledge the First Nations.

7
00:01:07.000 --> 00:01:19.000
On whose land we currently reside. So that's the acknowledgement and I think now we can We're ready to start.

8
00:01:19.000 --> 00:01:22.000
I'm not supposed to read it's outload or But you can but normally it's on the screen.

9
00:01:22.000 --> 00:01:32.000
But if you'd like to keep moving forward. And now I should click here.

10
00:01:32.000 --> 00:01:40.000
So it works.

11
00:01:40.000 --> 00:01:41.000
Hey, hello everyone. I'm glad that they can, be with you this or indirect way.

12
00:01:41.000 --> 00:01:53.000
Of course I would have much preferred to be able to participate in an active

13
00:01:53.000 --> 00:01:58.000
Manner but that wasn't possible for me. I apologise for that.

14
00:01:58.000 --> 00:02:11.000
I will directly jump into this presentation. About using comparative survey data to describe the religious heterogeneity of societies.

15
00:02:11.000 --> 00:02:20.000
That will show you a very small and simple benchmarking exercises, nothing complex really.

16
00:02:20.000 --> 00:02:29.000
So I hope it won't be easy enough to follow even this virtual presentation. The background question comes from the circumstance that repeated comparative survey programs are with us for a very long time now.

17
00:02:29.000 --> 00:02:52.000
So they prostitute a massive database. And that invites the use for informing about. Very large trends at a very high level of abstraction really.

18
00:02:52.000 --> 00:03:01.000
So survey programs were built basically as tools for analyzing the social mechanics, mechanics of change.

19
00:03:01.000 --> 00:03:15.000
And of course for comparison across countries because they allow the analysis of micro multi level associations in time series cross sectional designs and they are extremely useful for that we know that.

20
00:03:15.000 --> 00:03:26.000
And our question is if they are also good for just looking at rather simple but very highly abstracted. Measures of the social structure.

21
00:03:26.000 --> 00:03:27.000
Please.

22
00:03:27.000 --> 00:03:39.000
For which we would usually be using data from official statistics of some form or from expert surveys, which is what I will be using as a benchmark.

23
00:03:39.000 --> 00:03:42.000
Later ends up presentation.

24
00:03:42.000 --> 00:03:53.000
You possible advantage of, deriving, high level destructive aggregate measures from survey data is.

25
00:03:53.000 --> 00:04:06.000
That surveys. More detailed they have information that the statistics may not have but which is still social science researchers, they are also often more timely.

26
00:04:06.000 --> 00:04:19.000
The possible at disadvantage is that they would be less precise and reliable because they are obviously sample based and samples.

27
00:04:19.000 --> 00:04:33.000
Can be of variable quality. We also may have methodological changes and data collection instruments which should be less.

28
00:04:33.000 --> 00:04:46.000
Prominent or less frequent with official statistics. And that's not always so easy to

29
00:04:46.000 --> 00:05:02.000
Detect these changes in the documentation we have. And there's the angle that we can, use surveys and other data in the complementary fashion to fill in the country in time coverage gaps at one or the other might have.

30
00:05:02.000 --> 00:05:07.000
But for that I would need to be of comparable quality of course.

31
00:05:07.000 --> 00:05:17.000
I application deals with the measurement of fractionalisation at a higher level. Property of societies.

32
00:05:17.000 --> 00:05:30.000
That's the absence of homogeneity. In regard to categories of social belongings. So We have famous applications of that in the field of ethnic fractionalisation.

33
00:05:30.000 --> 00:05:41.000
Introduced as a term into the economics field, Lesina and co-authors. And they use this to predict.

34
00:05:41.000 --> 00:05:51.000
The wealth of the development status or trajectory of countries, especially in the desert. Thought world.

35
00:05:51.000 --> 00:06:02.000
Very often with the outcome that more ethnicically fraternized countries would have less promising development pathways in economical terms.

36
00:06:02.000 --> 00:06:12.000
So same kind of measure as also popular and political science where we use it to assess refractionisation of party systems.

37
00:06:12.000 --> 00:06:21.000
So how voters or citizens are distributed over parties. The more parties we have and some more evenly voters distributed overseas.

38
00:06:21.000 --> 00:06:29.000
The lesser is potentially the stability of governments and moral privatization might arise. But that's not a fair.

39
00:06:29.000 --> 00:06:39.000
Very strong association they can also be a different pathways to that. Unless they, they trust they could be, etc.

40
00:06:39.000 --> 00:06:47.000
Falsize the typically hypothesis for that. Our focus however is on religious fractionalization.

41
00:06:47.000 --> 00:06:57.000
Which I see as vantage that it is conceptually critically much cleaner so maybe an ethnic fractalisation was ethnicity, we really don't know.

42
00:06:57.000 --> 00:07:11.000
How to define and it's certainly relevant in its own right. With the processes of secularisation and migration going on in many societies.

43
00:07:11.000 --> 00:07:18.000
It is a research question. What increasing

44
00:07:18.000 --> 00:07:26.000
What consequences increasingly religious diversity might have really for many things. They will not go into this for reasons of time.

45
00:07:26.000 --> 00:07:39.000
We need to, fractionalization also is a component of the IRS, you know, ethnic Factoryization, And a further advantage is that it's easily computed from available data.

46
00:07:39.000 --> 00:07:47.000
All we need is the shares of the nominations in a given society and we can get that from surveys.

47
00:07:47.000 --> 00:08:01.000
All from other sources like Census's official statistics. The measure we use. Well, that is the complement of the rather well known half in the Verseman andics of concentration.

48
00:08:01.000 --> 00:08:11.000
Which is computed as easy. Some of these create shares. The groups.

49
00:08:11.000 --> 00:08:16.000
In the society have of something particular.

50
00:08:16.000 --> 00:08:32.000
So we have after Frederinization index equal to one minus 8 H. And then after this grabs the probability that 2 randomly chosen individuals belong to different groups.

51
00:08:32.000 --> 00:08:43.000
That has a minimum of 0. Which it can only, obtain if. K is equal to one, only one groups exist.

52
00:08:43.000 --> 00:08:55.000
At the maximum of one minus one divided by k. If all groups have the same share really then

53
00:08:55.000 --> 00:09:08.000
We have maximum fractionalisation in our society. So the number of groups is a constant here that the more groups the more fractionalisation we can have.

54
00:09:08.000 --> 00:09:13.000
Bill F is always lower than one.

55
00:09:13.000 --> 00:09:25.000
Interestingly, the HAVING DULTI-FORTIONATION MAYORS mainly react to variation in the large shares in due to the scaring of the proportions.

56
00:09:25.000 --> 00:09:34.000
Obviously, I saw that small groups don't play a very relevant role role in these indices.

57
00:09:34.000 --> 00:09:54.000
Our benchmark datasets. Oh, expert created data series or data sets. And these experts have drawn on different sources, for the, index of religious fractionalization.

58
00:09:54.000 --> 00:10:10.000
Yes, he detected his slides. The ethnic factor of relation one. The source is a bit unclear that when he, actually went back to Encyclopaedic work even back to the 19 sixties.

59
00:10:10.000 --> 00:10:34.000
And then updated the information about religious groups they get from this for a reference here of 2,000 so the actual data that they are a fraternalization index about religious denominations is from probably related to the 19 eighties, 19 nineties.

60
00:10:34.000 --> 00:10:41.000
Larger project is the composition of religious and ethnic groups projects in brief crack.

61
00:10:41.000 --> 00:10:53.000
We are also using tertiary data. Precompiled by others, 3 main sources, Britannica, CIA, World Factbook and World Almanac booked effects.

62
00:10:53.000 --> 00:11:08.000
And they have again. Reconciliation process by experts. I do an annual project and smoothing and I write this data from 2,013.

63
00:11:08.000 --> 00:11:19.000
Finally there is a world religion project. Which I like the others doesn't go to books but directly to the National Census and supranational databases.

64
00:11:19.000 --> 00:11:29.000
But again, they do expert reconciliation, put the data into 5 year intervals and their latest data point than is from 2,010.

65
00:11:29.000 --> 00:11:34.000
So sorry they just said we use.

66
00:11:34.000 --> 00:11:43.000
The accurate object of interest originally is the church and tendency. And we just change pulled European dataset.

67
00:11:43.000 --> 00:11:54.000
That was created by Bielkati and co-authors. It has aggregates derived from the Jupermenters from European Social Survey.

68
00:11:54.000 --> 00:12:07.000
From the European value study from the International Social Survey Program and from the World War 2 Survey. That implies that obviously we will focus on Europe and not the world.

69
00:12:07.000 --> 00:12:15.000
The other programs, the expert based and the surveys have worldwide coverage, Cara only has European coverage.

70
00:12:15.000 --> 00:12:28.000
We will focus on that. We will also focus later on the time window of 2,000 to 2,020 because I just should be enough to get a general impression of the dignity of these things.

71
00:12:28.000 --> 00:12:44.000
In Carpet we have 45 European countries. Not all covered equally well with and a long time serious but again I would just say we will only start in the year of 2,000 going to 2,016 when

72
00:12:44.000 --> 00:12:54.000
Dataset. The usual things if you work across data sets I would just skip over this

73
00:12:54.000 --> 00:13:06.000
Sake of time and we will have 3 very simple analysis questions here. One is how well do the survey estimates, survey based estimates that is agree among themselves.

74
00:13:06.000 --> 00:13:18.000
He's a program effect on the dispersion on the levels. The second angle of looking at this is how well do the expert estimates.

75
00:13:18.000 --> 00:13:20.000
Agree among themselves. And finally, how well it is the survey and expert estimates agree with the other type.

76
00:13:20.000 --> 00:13:29.000
And I will show that in 3,

77
00:13:29.000 --> 00:13:30.000
Graphical presentations. So we will just be eyeballing these things and look for happening irregularly.

78
00:13:30.000 --> 00:13:54.000
This deviation across the data sources really and that will be it. But first before we do that, this is the distribution of the estimate that, 3 to the image mails in the survey database so just that you can see that it's Nicely distributed.

79
00:13:54.000 --> 00:14:08.000
Nothing wrong here really. And we jumped directly into I-borning the outcomes for the between survey based estimates.

80
00:14:08.000 --> 00:14:12.000
Check for the European countries.

81
00:14:12.000 --> 00:14:19.000
By the way, I had to include the word value study data from this, so it's only 4 sources.

82
00:14:19.000 --> 00:14:28.000
Because of WVS data had a coding oddity not in the actual world value survey park and the transform to the carpet.

83
00:14:28.000 --> 00:14:34.000
Data set which basically spoils them for this.

84
00:14:34.000 --> 00:14:45.000
So it's only 4 lines and the general impression you would hopefully get here is that most of these cells are rather nice in a way.

85
00:14:45.000 --> 00:14:58.000
They look closely nothing. But there are some deviations for example in Bulgaria we have rather large deviations or Denmark, Estonia.

86
00:14:58.000 --> 00:15:03.000
Iceland looks a bit wild.

87
00:15:03.000 --> 00:15:09.000
No way and train. Who they don't.

88
00:15:09.000 --> 00:15:17.000
Behave perfectly well obviously and the impression as well there is reason for deep pain investigation which we are not doing here.

89
00:15:17.000 --> 00:15:26.000
We also have these very nice pictures really Switzerland or They are tracking pretty well.

90
00:15:26.000 --> 00:15:30.000
Is there a program effect?

91
00:15:30.000 --> 00:15:45.000
On the level of the index estimate. Yes, slightly. So ESS estimates across countries. Tend to be a little higher than the others, but that was not always the case.

92
00:15:45.000 --> 00:15:55.000
And ISISB. Estimates tend to be a little lower but it's really not that much given the variation that we have seen here.

93
00:15:55.000 --> 00:16:06.000
But there's also little reason to expect that. Programming effect would be consistent over countries really. Because maybe except for the ESS.

94
00:16:06.000 --> 00:16:14.000
The consistency and the fieldwork methodologies cannot be.

95
00:16:14.000 --> 00:16:25.000
So perfect. Across countries really that they are any difference in their approaches with Kashmiri over the whole.

96
00:16:25.000 --> 00:16:28.000
Set of countries really.

97
00:16:28.000 --> 00:16:38.000
But now for the, expert based sources. 3 years. I mean, we have, if you remember, that was a Lazina that was in Craig.

98
00:16:38.000 --> 00:16:50.000
Database and the. Religion project database. We have different densities of projections, Alzheimer only has this one time point.

99
00:16:50.000 --> 00:16:58.000
Votual agent has 3. And Craig has these protected serious. And over all their trek.

100
00:16:58.000 --> 00:17:08.000
Acceptably well with the exception of a few countries, Belarus and Check here and Moldova.

101
00:17:08.000 --> 00:17:20.000
Wears Mandova here as Mandova. Don't look so nice and Alzheimer is a bit away from the other, very often and on top of that has a problem.

102
00:17:20.000 --> 00:17:31.000
Of being trusted. Single data point really. But it is actually the most popular in the literature, which is a bit weird.

103
00:17:31.000 --> 00:17:42.000
Now putting it all together and concluding really, we see that. Most observation cells that we have here.

104
00:17:42.000 --> 00:17:48.000
This survey data match the expert based data pretty well.

105
00:17:48.000 --> 00:18:01.000
Went often also lie in between. Oh, close to the survey data lines. And in the overall picture It rather appears.

106
00:18:01.000 --> 00:18:10.000
That sometimes it's a crack CEOs which is a bit farther away from the pegs really.

107
00:18:10.000 --> 00:18:17.000
And appears to be the outlier. For example in in Chechnya. Or in.

108
00:18:17.000 --> 00:18:23.000
Spain, Spain is here, but in Sweden it's similar.

109
00:18:23.000 --> 00:18:32.000
We have Krak being the Ottawa now to really on a new hook train again. Don't know why that is.

110
00:18:32.000 --> 00:18:41.000
Forgotten Norway and this least by the way. But always a bit diverse in terms of its measurement also.

111
00:18:41.000 --> 00:18:52.000
We don't know where that is, but it is worth noting and it invites our conclusions really and the conclusion is that there is no obvious way.

112
00:18:52.000 --> 00:19:05.000
In terms of which data type would be more consistent among themselves. Really, for estimating. Fractionalization in these societies.

113
00:19:05.000 --> 00:19:14.000
That we have of looked at here. And we cannot really for now explain these differences. That was not the intention here.

114
00:19:14.000 --> 00:19:18.000
It was just diagonalises for the moment.

115
00:19:18.000 --> 00:19:26.000
If we have learned anything systematic may be a role that crack is a bit of an outlier and that would measure further investigation.

116
00:19:26.000 --> 00:19:35.000
Certainly. Generally the trends seem to be described rather consistently by all the measures.

117
00:19:35.000 --> 00:19:43.000
So there's nothing that speaks against using any of them except Azina which cannot describe trends.

118
00:19:43.000 --> 00:19:59.000
Richard in a ready for Yorkshire. Further steps would then for developing this project further within of course be to try to drill a bit deeper into the reasons for these deviations.

119
00:19:59.000 --> 00:20:07.000
And find further data sources to bring into the picture. So that is it and I thank you very much and.

120
00:20:07.000 --> 00:20:17.000
Be glad if you would send any questions could have. Comments. Critical remarks, whatever to this contact address.

121
00:20:17.000 --> 00:20:21.000
Thank you again.

122
00:20:21.000 --> 00:20:33.000
You and And now we will move to, the second presentation and we'll have the questions at the end.

123
00:20:33.000 --> 00:20:36.000
Can I just share my screen?

124
00:20:36.000 --> 00:20:41.000
Yes, we hear you, but

125
00:20:41.000 --> 00:20:44.000
Yeah, if you would then.

126
00:20:44.000 --> 00:20:50.000
Yep.

127
00:20:50.000 --> 00:20:57.000
You probably know more than me. Okay. Yeah, we can see your screen. Please.

128
00:20:57.000 --> 00:20:58.000
And what is yours?

129
00:20:58.000 --> 00:21:03.000
Yeah. I'm afraid.

130
00:21:03.000 --> 00:21:07.000
Hi.

131
00:21:07.000 --> 00:21:14.000
Let me read. I'm sorry, I'm working at statistics Netherlands and we have very high security measures so I really have to

132
00:21:14.000 --> 00:21:22.000
Okay.

133
00:21:22.000 --> 00:21:35.000
And then.

134
00:21:35.000 --> 00:21:39.000
I'm sorry, it doesn't seem to work.

135
00:21:39.000 --> 00:21:47.000
It worked yesterday, but

136
00:21:47.000 --> 00:21:57.000
Yes, okay. I'm back in.

137
00:21:57.000 --> 00:22:06.000
And now I'm sharing mine. Screen.

138
00:22:06.000 --> 00:22:07.000
Yeah, we can see it.

139
00:22:07.000 --> 00:22:14.000
And now it should work, right? Yeah. Okay, sorry. So, thank you so much for, for attending, this session.

140
00:22:14.000 --> 00:22:22.000
My name is, Today I'm going to present about a study that we have been doing.

141
00:22:22.000 --> 00:22:28.000
We actually have been doing several studies on this. On how to use devices, in this case, accelerometers.

142
00:22:28.000 --> 00:22:43.000
Instead of or in addition to survey questions. And this is joint work with colleagues at Statistics Netherlands and Mike Ecompete and Anna Alevelt and also PhD at the Amsterdam Medical Center in the Wolf.

143
00:22:43.000 --> 00:22:51.000
And yours Mildred from Centre Data who actually did The building of the survey and sending out all these devices.

144
00:22:51.000 --> 00:23:02.000
So what are we looking at here? Typically in general, basically all countries use survey questions in order to assess physical activity.

145
00:23:02.000 --> 00:23:14.000
And we know that that is not the best method because of social desirability and the fact that people find it very difficult to estimate how active they are, their own physical activity.

146
00:23:14.000 --> 00:23:19.000
And this is a nice example to demonstrate that. So this is a study by Arik Epstein in the United States.

147
00:23:19.000 --> 00:23:44.000
And he has a survey which is a subjective measure and an objective measure which is an activity tracker and you see the differences here and basically when you look at the objective measure more people are inactive compared to answers to survey questions and you can imagine if you want to develop a policy you really want to know how many people are actually inactive and which people are

148
00:23:44.000 --> 00:23:54.000
inactive. So there are different ways to to use devices. People have their own like activity tracker smartwatches.

149
00:23:54.000 --> 00:24:04.000
So this is what we call people's own activity trackers. These are commercial devices. And you also have research great devices that you have to buy.

150
00:24:04.000 --> 00:24:15.000
They typically cost a couple of $100. or euros and they can be placed on the risk but also on the hip or on top of the lag.

151
00:24:15.000 --> 00:24:33.000
So what we did, we first did a lab study to investigate which sensor actually which type of the research great sensors worked best and we came to the conclusion that active poll was the best measure at least in the Netherlands because it is placed on top of the lac and in the Netherlands we do a lot of cycling so

152
00:24:33.000 --> 00:24:40.000
for us it is very important that we are able to detect whether or not people are on a bike on not.

153
00:24:40.000 --> 00:24:46.000
And also it's very nice of, this device is that they provide very nice feedback to respondents.

154
00:24:46.000 --> 00:24:56.000
So you see in blue if people are lying and for example red and purple are when people are highly physically active.

155
00:24:56.000 --> 00:25:04.000
So the goal of this research was to investigate whether or not activity tracker is feasible in national population monitoring.

156
00:25:04.000 --> 00:25:16.000
And we did a study in the probability based LIST panel. We had, first a screening question to ask whether or not people had their own activity checker, so smartwatch, Fitbit, etc.

157
00:25:16.000 --> 00:25:23.000
Then we did a survey in the Netherlands we used a squash survey which is a survey on physical activity.

158
00:25:23.000 --> 00:25:29.000
And then, we use this active pulse and you can imagine we don't have hundreds of active calls.

159
00:25:29.000 --> 00:25:40.000
So we had to do that in batches. We had about a hundred 50 active balls, so we send out in batches and then they came back again and then we send out another batch.

160
00:25:40.000 --> 00:25:49.000
And for the, the study what I'm going to be talking about, we had almost 500 people at their own device.

161
00:25:49.000 --> 00:25:55.000
We selected hundreds that own the fitbits and why we did that. So we wanted to investigate basically 2 things.

162
00:25:55.000 --> 00:26:02.000
So first, whether or not people are willing to enter the numbers from their own tracker into a survey.

163
00:26:02.000 --> 00:26:23.000
So for example, if we ask a number of steps taken in a day. If people were going to look this up at in their watch and then enter it into the survey and also if you own an activity tracker you know that you you can often offload uploads or download spreadsheets from an entire week or an entire

164
00:26:23.000 --> 00:26:41.000
month. And we also wanted to investigate whether people are willing to do that. Well, you can imagine that you have to, build, a protocol to do that like a step by step plan on how to download these spreadsheets.

165
00:26:41.000 --> 00:26:47.000
And we came to the conclusion that that basically only worked for Fitbit. So for example, Garmin, it was more.

166
00:26:47.000 --> 00:27:01.000
Burdensome and also Garmin more or less doubles the highly active minutes. So this is was already one of the problems that is very difficult to compare these commercial devices because they all use their own algorithms.

167
00:27:01.000 --> 00:27:13.000
So that's why we decided to focus on Fitbit only. And then we used 150 people that did not own an activity tracker as a control group.

168
00:27:13.000 --> 00:27:14.000
I

169
00:27:14.000 --> 00:27:18.000
And we also, did a second survey so that we had a time, 0 and time one.

170
00:27:18.000 --> 00:27:30.000
Measure of the survey. So as I mentioned, it is rather burdensome if you use these research great devices because you have to, send them to respondents and they have to wear it and then you have to get them back.

171
00:27:30.000 --> 00:27:40.000
So first we asked people to complete the survey, then we had to configure the device, the active poll, then we had to send out the invite the device with instructions on how to wear it.

172
00:27:40.000 --> 00:27:55.000
People started to wear it for a week. Then they stopped wearing had to send it back to us then we had to extract the data we had to clean the device recharge the device and then we could send it back in a new batch.

173
00:27:55.000 --> 00:28:10.000
So, we asked people first whether or not in so in the entire survey. Whether they were willing to donate their own activity tracker so from the people who own the tracker which was 7 708 people.

174
00:28:10.000 --> 00:28:18.000
Almost half of the people were willing to donate their data. Meaning that they were willing to enter their data into the survey.

175
00:28:18.000 --> 00:28:33.000
Some people wanted to donate only some of the data they have in their device and actually some people were not sure whether or not they were willing to do so and when it comes to uploading a spreadsheet about a third of the people who are willing.

176
00:28:33.000 --> 00:28:40.000
To do that. So a third of the people who own a tracker are willing to upload a spreadsheet.

177
00:28:40.000 --> 00:28:47.000
Then we looked at who owns an activity tracker, so there is no effect of gender, but I is one of education and age.

178
00:28:47.000 --> 00:28:55.000
So people with high education and young people tend to have an activity tracker. And we also looked at other predictions.

179
00:28:55.000 --> 00:29:21.000
So people with high income, people with a high, general health people who have a wish score high on tech those people tend to have an activity tracker and people who have physical limitations tend to not have an activity tracker and you can imagine that if you want to use these devices in research that you will have a bias.

180
00:29:21.000 --> 00:29:37.000
Based on who owns the device. Then we compare it, the survey results, with the device results to see if some people tend to overestimate their own physical activity in a survey or underestimate.

181
00:29:37.000 --> 00:29:48.000
And you can see that here. So according to the active poll, when you look at people who are not active based on the active bodies, but they are active based in the survey.

182
00:29:48.000 --> 00:29:58.000
Those are 64 people. But actually we had more people who were not active in the survey, but they were according to the device.

183
00:29:58.000 --> 00:30:04.000
So people tend to overestimate their own physical activity.

184
00:30:04.000 --> 00:30:26.000
Well, if we compare these 4 groups, you can see here that people who are not active tend to have tend to be older than people who overestimate their general health and also when we look at physical limitations you also You see it the difference here.

185
00:30:26.000 --> 00:30:38.000
So people with physical limitations tend to overestimate. Other physical activity. Then we try to predict who meets the physical activity guidelines.

186
00:30:38.000 --> 00:30:52.000
So the World Health Organization, has, developed physical activity guidelines, which is basically the to be physically active 150 min per week or more.

187
00:30:52.000 --> 00:30:59.000
And you see, differences here. So, when you look at all these, predictors.

188
00:30:59.000 --> 00:31:10.000
Only education is a significant one while using the active poll. Well, with the squares survey, none of all these predictors are significant.

189
00:31:10.000 --> 00:31:21.000
And that basically means that it is very difficult to predict physical activity because there is not many predictors that that you can use.

190
00:31:21.000 --> 00:31:29.000
Then we wanted to investigate. The amount of bias we have in. In research on physical activity.

191
00:31:29.000 --> 00:31:37.000
And, yeah, you see that, Many. Of these predictors are significant.

192
00:31:37.000 --> 00:31:48.000
So you have bias. You have the young people. You have the highly educated people with high income, female people who don't have physical limitations, who do not own the latest technologies.

193
00:31:48.000 --> 00:32:05.000
Were at the forefront of technological developments and who does do worry about a privacy. I'm sorry, having a significant effect on willingness to participate but we did not find an effect of general health.

194
00:32:05.000 --> 00:32:12.000
So death was remarkable. Then when we look at data donation of people's own trackers, so again we have this flowchart.

195
00:32:12.000 --> 00:32:20.000
We had the the first measure, the squash one. And then, people were the active poll, for at least 4 days.

196
00:32:20.000 --> 00:32:29.000
But, and we all also ask people to donate their data. So either by downloading a spreadsheet or by manual entry.

197
00:32:29.000 --> 00:32:42.000
So what is the information that is typically in these, smartwatches, number of steps, total distance, number of active minutes, total calories burns, average health, and a duration of sleep.

198
00:32:42.000 --> 00:32:57.000
And you see that, 76%. Of the people. Who, own a device, shared the information, which is 70% of the entire sample.

199
00:32:57.000 --> 00:33:04.000
So if you wanted to do this type of study on the general population, you end up with 70% of the entire sample.

200
00:33:04.000 --> 00:33:22.000
And what we saw does information that was often admitted was average heart rate and sleep. Either because people could not find that information maybe because a device didn't have that opportunity or they didn't want to share.

201
00:33:22.000 --> 00:33:39.000
But what we always had was number of steps. Which is basically, the variable that all devices have and also what we have seen is very similar between a research great device and a people's own activity tracker.

202
00:33:39.000 --> 00:33:44.000
So then when we looked at the data uploads for the uploading the spreadsheet.

203
00:33:44.000 --> 00:34:04.000
Again, many variables that can be. In these spreadsheets. From the 61 fitbit users 43 were willing to upload a spreadsheet and then actually only 34 uploaded the spreadsheet of which only 26 were usable.

204
00:34:04.000 --> 00:34:23.000
Some people uploaded the wrong week, some people uploaded an empty spreadsheet and some people did not have any active poll data so we couldn't compare them with so from all the people who have their own tracker and who we wanted to, have, their spreadsheet only 26 were

205
00:34:23.000 --> 00:34:36.000
usable. So this meant to us that data donation via a spreadsheet is not a usable option if you want to monitor physical activity in the general population.

206
00:34:36.000 --> 00:34:45.000
So what, what was our conclusion that it is feasible to do national monitoring with research great devices, but participation is highly selective.

207
00:34:45.000 --> 00:34:59.000
But bear in mind participation is also highly selective when you do a survey. So we really have to compare whether or not the selectivity is higher when you use devices compared to survey questions.

208
00:34:59.000 --> 00:35:08.000
The difference in serving and device when it comes to physical activity, we did not find a large effect of personal characteristics only.

209
00:35:08.000 --> 00:35:14.000
Age in the sense that all the people tend to overestimate their physical activity in the survey.

210
00:35:14.000 --> 00:35:22.000
And we found that people are willing to donate to the the data from their own tracker. So entering the data in the survey.

211
00:35:22.000 --> 00:35:33.000
So just via regular survey questions is far better than asking them to upload a spreadsheet. It is too difficult, it is too much dependent on, these, different brands.

212
00:35:33.000 --> 00:35:45.000
And, yeah, people find it too difficult to do it. And then of course what is the goal of this session comparison of the survey results and the research great advice and the own tracker?

213
00:35:45.000 --> 00:35:52.000
And still to be done we have worked on it actually in in the last year but it is very hard.

214
00:35:52.000 --> 00:36:01.000
And we came to the conclusion that comparing survey results and device results is not really. A possibility because they measure different things.

215
00:36:01.000 --> 00:36:16.000
So in the survey you typically ask about sports. Or working to work or cycling, etc. Well, You If you have a device, it actually measures, for example, vacuum cleaning, very, very small activities that you do not take into account into a survey.

216
00:36:16.000 --> 00:36:29.000
So the device is probably more accurate but it is very difficult to compare the survey with the research great device.

217
00:36:29.000 --> 00:36:35.000
And when you compare the research great you flies with people's own trackers with it comes to number of steps they easily match.

218
00:36:35.000 --> 00:36:50.000
So that's great. When it comes to other variables for example number of active minutes it becomes very difficult because as I mentioned before Garmin tend to double the highly active minutes.

219
00:36:50.000 --> 00:36:58.000
So you cannot really compare the research great device with your own tracker, let alone different brands of commercial trackers.

220
00:36:58.000 --> 00:37:17.000
So that's very hard. The easy thing is that We came to the conclusion that if people have their own activity tracker, you probably don't have to ask all these questions about physical activity because people who have their own tracker do abide like in 99% of the time to the.

221
00:37:17.000 --> 00:37:24.000
To the guidelines of physical activity being, being active 150 min or more. So you have for whom is it worth?

222
00:37:24.000 --> 00:37:33.000
The easiest way is just to ask one single question, if you do a survey, do you have your own tracker?

223
00:37:33.000 --> 00:37:41.000
And maybe as the number of steps. But it is difficult to compare the survey results with the research great devices.

224
00:37:41.000 --> 00:37:51.000
And the research great devices are probably more, more accurate. But it is also very burdensome to having to buy them, sending them out, having all these batches.

225
00:37:51.000 --> 00:37:59.000
So, up till now, I think most, most countries still rely on survey questions when they measure physical activity.

226
00:37:59.000 --> 00:38:07.000
That's it. Thank you.

227
00:38:07.000 --> 00:38:19.000
Thank you. We will now move to our first in-person presenter, Sebastian is now.

228
00:38:19.000 --> 00:38:35.000
Terms.

229
00:38:35.000 --> 00:38:53.000
No.

230
00:38:53.000 --> 00:39:00.000
I need to. Yeah.

231
00:39:00.000 --> 00:39:12.000
So hello everyone, my name is Bustin Hunter from from the Institute for Employment Research in Germany and from the Institute for Employment Research in Germany and this talk is about the maximization of linkage and Germany.

232
00:39:12.000 --> 00:39:24.000
And this talk is about the maximization of linkage and this talk is about the maximization of linkage and consent using repeated requests and bearing framing.

233
00:39:24.000 --> 00:39:32.000
Administrative records on panel consent means that people agree to participate in further panel waves.

234
00:39:32.000 --> 00:39:44.000
I would like to go to the next slide, but it's not working.

235
00:39:44.000 --> 00:39:53.000
Just keep on restart. No, it's, I don't know whether it's the keys, maybe, or maybe sometimes the keys get stuck.

236
00:39:53.000 --> 00:40:03.000
You can take a break. Okay. But we won't take it out of your place. Okay, you sort this out, I just continue.

237
00:40:03.000 --> 00:40:11.000
And so the reason why I'm starting with this topic is that there's a new law in Germany and there are 2 new employment programs.

238
00:40:11.000 --> 00:40:30.000
And this is, yeah, some wage subsidies. So people who are long-term unemployed, yeah, could get back into work and at our institute we are evaluating these 2 employment programs and we look on the employability and the social participation of these.

239
00:40:30.000 --> 00:40:47.000
Persons. And we are evaluating this having different studies there qualitative and quantitative ones and we are also conducting their panel study where we interview these persons and that entered this employment program.

240
00:40:47.000 --> 00:41:07.000
And, yeah, we had some. Because constraints to conduct the study in so far as we can consider for the study in so far as we can consider for the relevant analysis only linked cases.

241
00:41:07.000 --> 00:41:18.000
And another constraint was that we only consider, yeah, these cases with panel consent. So meaning we did not have the option of a refreshment sample.

242
00:41:18.000 --> 00:41:30.000
So we really need to work with the base sample we have. And make the best out of this. Okay, so let's have a look on the linkage content rate literature.

243
00:41:30.000 --> 00:41:37.000
Some design aspects are there considered when you are setting up a survey. One is the placement of the question.

244
00:41:37.000 --> 00:41:49.000
It can be at the beginning, the middle or the end of the questionnaire and research throws and that you have obtained the highest linkage constant rates if the linkage consent question is at the very beginning.

245
00:41:49.000 --> 00:42:06.000
Questionnaire. Another aspect is the framing. So if you want to link the data to particular you raise an argument to the respondents because yeah the data are more worth and more meaningful or you can shorten the questionnaire or you can save some money something like this and also these arguments can be framed different differently.

246
00:42:06.000 --> 00:42:18.000
Nutrally or positively or negatively. However, there are rather inconsistent results.

247
00:42:18.000 --> 00:42:29.000
When it comes to panel consent, yeah, how is this typically obtained? As a linkage consent, there's only one question within the questionnaire and panel consent.

248
00:42:29.000 --> 00:42:39.000
The Pinner Consent request is typically at the end of the questionnaire. Yeah, so there people may say yes I want to attend further in the study or they yeah.

249
00:42:39.000 --> 00:43:00.000
Reject this. And there's one exception. A study conducted in Billefort in Germany and there we had only rather low panel constant rates of about 45% for the web mode and the paper and cancel mode and what we did there was and that we had some follow-up procedures by a postcard and via a

250
00:43:00.000 --> 00:43:08.000
follow-up call and due to that we could increase the panel considerate by 25 percentage points so this was quite successful.

251
00:43:08.000 --> 00:43:15.000
And, yeah, so inspired by these aspects I just mentioned. I derived the first research question.

252
00:43:15.000 --> 00:43:25.000
It is to what extent can the remaining share of non-consenters be reduced in terms of kernel consent and linkage consent within the first panel wave.

253
00:43:25.000 --> 00:43:31.000
So I will ask am several times within a panel wave for linkage and panel consent.

254
00:43:31.000 --> 00:43:46.000
Another relevant aspect is what are the determinants of panel consent and linkage consent. There are some mechanisms like people that have privacy concerns or have a rather bad lack of understanding or rather hesitant to link their data.

255
00:43:46.000 --> 00:43:56.000
On the other hand, people might think that linking data might improve the data quality or we are participating in further further ML rates and so this might.

256
00:43:56.000 --> 00:44:10.000
Yeah, increase the likelihood of consent and there might be also a pre-existing relationship to the survey vendor or the institution that conducts a service that might have positive or negative impacts and of course there are many other aspects like interviewer effects.

257
00:44:10.000 --> 00:44:28.000
So other research questions I have, what are the determinants of linkage and panel consent and considering these several requests what are the determinants of a change in the preference, yeah, to link the data or to obtain parent consent considering this first panel wave.

258
00:44:28.000 --> 00:44:33.000
I'm using data of the panel study I mentioned it's called quality of life and social participation.

259
00:44:33.000 --> 00:44:45.000
It was conducted the first wave when the Corona pandemic started in Germany in 2,020 and we realized about 17,000 and telephone interviews.

260
00:44:45.000 --> 00:44:52.000
So you see here, the design, how, panel consent and linkage consent was obtained on the right hand side.

261
00:44:52.000 --> 00:45:05.000
You see panel consent was asked first at the end of the interview. Secondly, with a postcard and then in a third step by follow up phone call.

262
00:45:05.000 --> 00:45:23.000
We have 2 employment programs. On the 16 E program, this takes up 2 years. Then we have some step groups and then we have also the 16 I program where the employment program could take up to 5 years and we have a different design there.

263
00:45:23.000 --> 00:45:33.000
And within the 16 E panel, I did an wording experiment. So we have 2 groups and the first group and shorter duration was mentioned first.

264
00:45:33.000 --> 00:45:37.000
That's argument for linkage. The end of the questionnaire, the non-consenters received another argument.

265
00:45:37.000 --> 00:45:51.000
I higher meaningfulness if they link their data. And the third step was then again a postcard or a phone call so 4 attempts within a panel wave or together.

266
00:45:51.000 --> 00:46:00.000
And in the second group, the other wording, it was reversed the order. So first this meaningful argument, then the duration argument for the upcoming waves.

267
00:46:00.000 --> 00:46:09.000
For the panel Nonkon Centers, they receive the same argument, there's meaningfulness, and then the third step again, Costcat and phone call.

268
00:46:09.000 --> 00:46:15.000
In the 16 I panel, the design was different and there we had an obligatory linkage consent. What does it mean?

269
00:46:15.000 --> 00:46:25.000
It means if people do not provide linkage consent, there is no interview. So the interview was aborted and when they respondents were confronted with this.

270
00:46:25.000 --> 00:46:40.000
They received a follow-up item. I call it reconsideration. So this was explained this situation and some people said okay I don't want to have this into you was about it and others spontaneously spontaneously agreed.

271
00:46:40.000 --> 00:46:47.000
And in a third group there were those and that could be convinced and that they could be reconstructed a couple of days ahead.

272
00:46:47.000 --> 00:46:55.000
And some of them change then their mind away again in this year. Slope of request. So this is the design.

273
00:46:55.000 --> 00:47:20.000
And here you see the outcomes for the panel consent. So in the first step the panel consent rate was already quite high was 95%.

274
00:47:20.000 --> 00:47:28.000
Also provided panel consent so altogether we had a significant increase in a pennil concentrate of about 98%.

275
00:47:28.000 --> 00:47:37.000
Let's not look first on the linkage content within this. First experimental group of the 16 E panel.

276
00:47:37.000 --> 00:47:44.000
We have here also a very high base concentrate of about 97% with the first request the second request.

277
00:47:44.000 --> 00:47:52.000
There we had 45% of Consenters. The postcard was also not really successful with 3%.

278
00:47:52.000 --> 00:48:04.000
Consent and the follow-up phone call had 10% consent so all in total here we have 52% where they changed the mind and became also commenters.

279
00:48:04.000 --> 00:48:10.000
And the other experimental group with the other Order of the framings, the results are quite similar.

280
00:48:10.000 --> 00:48:23.000
96% is a base consent rate, then 50% changed the mind. Here where we had this meaningfulness argument again among this group of non Yeah, loan centers of final consent.

281
00:48:23.000 --> 00:48:36.000
It was only 8% because it was a selective group and the same argument which was not really Yeah, there's surprising the poster didn't work at all and this follow-up phone call also led to a 21%.

282
00:48:36.000 --> 00:49:01.000
And the content so all together 55% additionally provided panel consent. Okay, and here we have these panel with is an obligatory consent by definition only these providing panel consent can participate so it's by definition 100% here to better have an impression about the dropout I consider in this.

283
00:49:01.000 --> 00:49:15.000
Slide here also the partial interviews so we have here in the end 97% linkage content so the dropout due to this obligatory panel consent is about 3.2% here.

284
00:49:15.000 --> 00:49:25.000
And about 18% changed their mind due to this repeated requests. So let's look here on the determinants of panel consent.

285
00:49:25.000 --> 00:49:33.000
This is a logistic regression. So in the first model you see the outcome of the first panel request of the questionnaire.

286
00:49:33.000 --> 00:49:41.000
So it's one if people provide a panel consent. And in the second model, only the refusers are considered.

287
00:49:41.000 --> 00:49:50.000
So those that said no and the first request so we have here only very small case numbers 260 and in the first model it was 5,500.

288
00:49:50.000 --> 00:50:05.000
And yeah, and the second model I mean one means that people change their mind so Yeah, so, we have here some basic variables that are typically considered and at the bottom or other specific ones that are considered in this survey.

289
00:50:05.000 --> 00:50:13.000
And we see as an outcome that people with a migration background have a lower likelihood to provide panel consent.

290
00:50:13.000 --> 00:50:24.000
Or respondents that are interested in the survey who are rather intrinsically motivated also tend to provide panel consent.

291
00:50:24.000 --> 00:50:39.000
For those that change their mind, we see that this appears rather for people with a rather medium education. Or people that have a better, yeah, language skills, they are also more likely to change their mind and to provide.

292
00:50:39.000 --> 00:50:49.000
Panel consent. So now we switch to the other group. This was the 16 E panel. Now you see the 60 I pan on there the panel consent outcomes.

293
00:50:49.000 --> 00:51:01.000
We have a rather similar pattern in so far that also migration has here a negative impact. A respondent interest also has here a positive impact and we see a positive influence for people that are unemployed.

294
00:51:01.000 --> 00:51:29.000
So people who are unemployed or rather provide panel consent. This would be an indication that this pre-existing relationship with a survey institute or the institute that conducts the survey can have a positive effect and we see also a negative effect of some index of income missing so those that aren't providing information about the income who are rather hesitant

295
00:51:29.000 --> 00:51:38.000
to provide, yeah. Sensible data don't want to participate further in the study.

296
00:51:38.000 --> 00:51:52.000
Lastly, and we look here on the linkage content outcomes of the logistic regression. And we see here also a negative effect impact of migration also for these mind changers.

297
00:51:52.000 --> 00:52:14.000
We see also, a negative impact on the respondents language skills, also for these mind changes and we have positive effects in year for the unemployed people, people that provide a panel consent or typically panel consent and then good consent are highly correlated as they are some indicators of yeah.

298
00:52:14.000 --> 00:52:21.000
For operating also work motivation has a positive effect and the, yeah. Respondents ability to understand questions.

299
00:52:21.000 --> 00:52:38.000
Yeah, so this are the results to conclude. If we ask people several times within a panelway for, yeah, panel consent or language consent, it leads to a change of the mind for many respondents.

300
00:52:38.000 --> 00:52:57.000
So the peloton center rates could be improved significantly by 57% with this second interviewer administered request we had an increase of about 44% for this second request in the 16, a panel and also some of these obligatory code consent, this can work.

301
00:52:57.000 --> 00:53:07.000
The dropout was not that high with about 3%. But I did not find any significant effects of the framing of these different wordings.

302
00:53:07.000 --> 00:53:18.000
Mmm, yeah, and there were also some indications when cancel the determinants for privacy concerns or lack of understanding or a pre-existing relationship with the institution.

303
00:53:18.000 --> 00:53:32.000
And I have to mention that all these yeah results are dependent on the server mode on the kind of linkage request, whether you are pressing several times in which mode and of the population, how it's constructed.

304
00:53:32.000 --> 00:53:42.000
So this was only a katty's duty. Yeah. Telephone study with a special population of these long long-term unemployment persons.

305
00:53:42.000 --> 00:53:51.000
And if you ask yourself whether you would, yeah, conduct these several requests, I think it depends on how important for you as linkage.

306
00:53:51.000 --> 00:54:00.000
Yeah. Does it make sense for you to have nonlinear cases to spend money on this? Or how important is for you panel consent if you want to a a panel study.

307
00:54:00.000 --> 00:54:19.000
But yeah. Only have a cross sectional study due to a lack of panel consent. Yeah, it's also a question of money I would say so it's extra effort that you can yeah bargain with a survey vendor or if it's online probably not additional costs there are but yeah of course to every burden so you have to consider

308
00:54:19.000 --> 00:54:27.000
different aspects. So yeah, thanks for listening. That's it.

309
00:54:27.000 --> 00:54:44.000
Thank you. You hope for the technical difficulties. Great ease. So now we will move to the fourth presenter and this would be Degrant.

310
00:54:44.000 --> 00:54:50.000
So if there I might we need to

311
00:54:50.000 --> 00:54:53.000
Can I just share my screen now?

312
00:54:53.000 --> 00:54:55.000
No, not yet. We are having some difficulties.

313
00:54:55.000 --> 00:55:03.000
Okay. That's it.

314
00:55:03.000 --> 00:55:07.000
Yep.

315
00:55:07.000 --> 00:55:08.000
Oh, okay. And.

316
00:55:08.000 --> 00:55:11.000
Okay, we can see your presentation now.

317
00:55:11.000 --> 00:55:16.000
But how do I, switch the slides?

318
00:55:16.000 --> 00:55:19.000
Are you sharing your screen?

319
00:55:19.000 --> 00:55:24.000
No.

320
00:55:24.000 --> 00:55:26.000
So this might not be him. No, that's it. But you cannot you cannot move the slides.

321
00:55:26.000 --> 00:55:30.000
Yeah, it's not me. I don't I don't know how to do that.

322
00:55:30.000 --> 00:55:33.000
You should be.

323
00:55:33.000 --> 00:55:36.000
I mean there is no button or something. Thank you.

324
00:55:36.000 --> 00:55:44.000
You should be on your keyboard you should be able to make the slides forward like you normally would on a zoom.

325
00:55:44.000 --> 00:55:48.000
That only works if you don't share the screen. I believe this is how I did it.

326
00:55:48.000 --> 00:55:57.000
Oh, we need to stop sharing. Pardon me.

327
00:55:57.000 --> 00:56:01.000
Okay, and now

328
00:56:01.000 --> 00:56:02.000
Okay.

329
00:56:02.000 --> 00:56:10.000
Can you sorry, can you move up the top it says you are viewing?

330
00:56:10.000 --> 00:56:11.000
It works now. Can everybody see it?

331
00:56:11.000 --> 00:56:12.000
It works now, yeah. Hang on. I want to get rid of that. Oh, that's right. Oh, okay.

332
00:56:12.000 --> 00:56:17.000
Yes, we can say it then. OK, the floor is yours and it works.

333
00:56:17.000 --> 00:56:30.000
See everything? Okay. So I would like to present to you a little overview of our work in progress of our current research project we're working on since last year.

334
00:56:30.000 --> 00:56:35.000
The title is Internet and Media Exposure and the Association of Attitudes during the COVID.

335
00:56:35.000 --> 00:56:55.000
19 pandemic. So the basic substantial idea is the focus on attitudes, especially towards health protective behavior like the attitudes towards wearing masks or towards the measures or the risk assessment of infecting one self with COVID.

336
00:56:55.000 --> 00:57:07.000
And the interplay with the epidemiological data. So can you see is an increase in risk assessment or the satisfaction with the government handling with the pandemic.

337
00:57:07.000 --> 00:57:18.000
Can you see a an effect maybe in infection activity but also the interplay with the social media environment.

338
00:57:18.000 --> 00:57:29.000
So can you see? Does infection activity affect the way people communicate on social media and does this in turn.

339
00:57:29.000 --> 00:57:38.000
Affect the the attitudes towards several indicators regarding the pandemic and do you have a feedback loop to infection rates, something like this.

340
00:57:38.000 --> 00:57:53.000
This is the basic substantial idea. And to do so, we have the basic methodological idea is to merge several survey data sets from anything that we could get our hands on.

341
00:57:53.000 --> 00:58:18.000
So we collect all the survey data collected. In Germany so we consult the typical big large representative house household panel studies but also the smaller ones we have some regional corporations with their regional television stations to get some daily, daily questionnaires, about the pandemic.

342
00:58:18.000 --> 00:58:26.000
But we also include, epidemiological data in the sense of infection activity, the death toll or.

343
00:58:26.000 --> 00:58:37.000
The 7 day incidents, on a regional scale. But also macroeconomic data like the regional unemployment rates for example or the share of migrants.

344
00:58:37.000 --> 00:58:49.000
And the thing I will talk mostly about today is the social media context. So we collected several relevant relevant tweets with containing the related search terms.

345
00:58:49.000 --> 00:58:55.000
So on this slide here you can see the the periods you're covering so far, but it's not completely complete yet.

346
00:58:55.000 --> 00:59:04.000
So you see the the survey data from the different, which many of you probably already, mostly know.

347
00:59:04.000 --> 00:59:09.000
So we have the, we use, this panel, we use the socioeconomic panel, we use the Corona study done.

348
00:59:09.000 --> 00:59:26.000
The Manheim Corona study, the National Education Panel. And so on. You can see them here, but we haven't covered any of everything's completely so far since we're dealing with with a lot of data sets and finding the relevant variables.

349
00:59:26.000 --> 00:59:41.000
And on different databases and some days we're dealing with more than 100 data sets per study and you know data documentation can be a little bit bumpy sometimes so it's really time consuming.

350
00:59:41.000 --> 00:59:49.000
So, this is, yeah, the thing we are working most on at the moment.

351
00:59:49.000 --> 01:00:05.000
For the epidemiological data we use the pandemic and activity from the Robert Cock Institute, which is kind of our national epidemiological institute that is gathering most infection related data which is publicly available.

352
01:00:05.000 --> 01:00:19.000
And we use the infection rates, the the death toll, this on the 7 day incidence of the rates but also the vaccination rollout.

353
01:00:19.000 --> 01:00:20.000
Yeah.

354
01:00:20.000 --> 01:00:36.000
And for social media data, we used, a web scraping technique. We used a Python package to web scrape, different tweets from Twitter, Twitter posts, containing the route, the relevant keywords, which I will talk about in the next slides.

355
01:00:36.000 --> 01:00:56.000
But there was a problem. There is the The most actual and frequently used package to get tweets without the very costly Twitter inter internal Twitter API premium account that was used by Python.

356
01:00:56.000 --> 01:01:05.000
They had a breakdown like in early 2023. Everybody was using normally this package.

357
01:01:05.000 --> 01:01:11.000
I was also in contact with the person who was writing this package. It wasn't possible anymore because of the, you know.

358
01:01:11.000 --> 01:01:27.000
The new leadership of Twitter and they They restricted access of. of external, Data collection approaches so we couldn't use it anymore.

359
01:01:27.000 --> 01:01:36.000
We get got our hands on on already a lot of data from from Twitter but only for some several key words.

360
01:01:36.000 --> 01:01:48.000
Not all of them. We had a really big list of different things that we wanted to collect, but we could only Get our hands on on like a one of them, which I will go on briefly later.

361
01:01:48.000 --> 01:01:56.000
So we have to think about how can we include other. Other social media data so we came up with the idea.

362
01:01:56.000 --> 01:02:13.000
So you use certain telegram channels which We know before, that they have a certain already have a certain stance or certain position towards different elements and aspects of the pandemic.

363
01:02:13.000 --> 01:02:24.000
So we were using far right and conspiracy groups as you can as defined and already looked in at by the college.

364
01:02:24.000 --> 01:02:29.000
And we also used, yeah, sentiment analysis, which I will also Go briefly into what which problems may arise.

365
01:02:29.000 --> 01:02:54.000
But when doing so using the tweets. And of course all of this because this talk is about the methodological challenges there are some challenges which arise and some of them are very general which arise in almost any research project but I will discuss them just very briefly.

366
01:02:54.000 --> 01:03:03.000
So for example, the very typical question of if your analysis steps are they data or be read-driven.

367
01:03:03.000 --> 01:03:11.000
This is especially relevant for example with variable selection and the exploration of the timeframes and so on.

368
01:03:11.000 --> 01:03:32.000
Since we are very dependent on the data structure and rather than a theoretical assumption. So For example, when you look at, items, like mask wearing, for example, they, they almost have no variants, over, the very different studies that we, that we looked in.

369
01:03:32.000 --> 01:03:37.000
So we have to choose another one and this is not based on theory but based on the data but also there are many variants related problems for example.

370
01:03:37.000 --> 01:04:03.000
So the time points so you have different studies and the difference, the different data doesn't have the same variance on, on the same time points so you have to Think about, for example, like analyzing lagged effects rather than contemporary effects and so on.

371
01:04:03.000 --> 01:04:10.000
There is of course, validity, and reliability problems. Especially when you deal with social media data.

372
01:04:10.000 --> 01:04:19.000
I want to get into more into the reliability or replicability. So we wanted to use the reliability or replicability.

373
01:04:19.000 --> 01:04:24.000
So we wanted to use web scraping because of it. So we wanted to use web scraping because of its advantage to replacing because of its advantage to replicate the data collection steps.

374
01:04:24.000 --> 01:04:29.000
Very easily but the you know this dependence of this more or less voluntary or involuntary cooperation with private enterprises.

375
01:04:29.000 --> 01:04:54.000
Such as Twitter. Can lead to a sudden change of yeah these large corporations. Just, change the the contact or the access to to the data so it cannot be replicated in the same way.

376
01:04:54.000 --> 01:05:16.000
As you want but there's And I'm sure if I should talk about this now elected because there are many inter-related things we we need to think about for example we were using Difference, keywords and the way you use, the Python package to get the tweets, you know, if

377
01:05:16.000 --> 01:05:26.000
you If you have a several keywords for example for Corona, COVID you use all the synonyms that is that is that is related to pandemic.

378
01:05:26.000 --> 01:05:35.000
If you combine them with an or operator And we got really unreliable results, you know, so we have like 20 search terms.

379
01:05:35.000 --> 01:05:42.000
And for each inquiry when you press one of the code to get all the tweets sometimes you get 500 tweets for a certain city in a certain time frame.

380
01:05:42.000 --> 01:05:52.000
Sometimes you get 270. So there's almost a 50% difference in the number of tweets.

381
01:05:52.000 --> 01:06:01.000
So we try to get around that and. We were writing a script that is looping over every search term individually.

382
01:06:01.000 --> 01:06:02.000
Yeah.

383
01:06:02.000 --> 01:06:10.000
So you only look for Corona, then you get all the data, then you only look for the such term COVID.

384
01:06:10.000 --> 01:06:15.000
19 and get the data. And in the end, you pick out the unique tweet IDs.

385
01:06:15.000 --> 01:06:24.000
And there we got more or less good reliable results in the sense of the the amplitude with the variance in a number of tweets we got.

386
01:06:24.000 --> 01:06:44.000
Were range between you know from 530 and 510 for example so there's only 20 tweets that we, that made the difference in this in the different approaches compared to using the or operators.

387
01:06:44.000 --> 01:06:53.000
And we were in contact again with the with the person who wrote the package but This guy said this is the only way to get around it.

388
01:06:53.000 --> 01:07:03.000
So. This is the problem with reliability. We have of course the the social media problem of representativity.

389
01:07:03.000 --> 01:07:10.000
I will get to into that in the in the next slide again. Of course we have,

390
01:07:10.000 --> 01:07:29.000
That's You cannot infer the demographics, from, certain platforms because they don't have, the same inclusion probability, then the general problem, the general population. There are systematic biases.

391
01:07:29.000 --> 01:07:38.000
Of course, when you when you have to. Look at the certain practices and technological issues of a platform.

392
01:07:38.000 --> 01:07:45.000
So if you use the number of followers as an indicator for relevance of the of the account.

393
01:07:45.000 --> 01:07:52.000
But how do you know how many of them are bots or how many of them are bought by private enterprises, for example.

394
01:07:52.000 --> 01:08:01.000
And the well the ethical issues both on a more general end on an individual level like who owns the data in the end.

395
01:08:01.000 --> 01:08:27.000
We are in this tourist, traditional gray area where we just assume that when you post something on the internet which is publicly accessible and which is stored already that you already gave the consent that somebody is maybe looking into the things that you that you said on the internet but well this is a very general problem in And if you do

396
01:08:27.000 --> 01:08:37.000
selfless science with social media data. So getting to the more, specific. Errors that might occur occur and using social media data.

397
01:08:37.000 --> 01:09:02.000
I want to to rely on this framework formulated by Sen and colleagues since they they they refer to this total survey error framework where they adapted this framework for for online platforms which I find really useful and I can only recommend so we get a common vocabulary and categories to think about everything.

398
01:09:02.000 --> 01:09:19.000
Dealing with this kind of data. So on the left side in blue we see errors that can occur. Well, regarding the platform, the platform affordances is this what I've already talked about in the sense of technological aspects of the of the platform.

399
01:09:19.000 --> 01:09:36.000
What we for example didn't look in because we yet. We are dealing for example with You look at Twitter data in a longitudinal.

400
01:09:36.000 --> 01:09:37.000
Yeah.

401
01:09:37.000 --> 01:09:45.000
Manner, but what if you use the indicator a number of likes for example what what if there was a systemic stomach change like a it's like a scale change more or less you know so what if instead of only you can press like now.

402
01:09:45.000 --> 01:09:48.000
So what if instead of only you can press like now you can press like now you can press like now you can press heart or you know, emojis or whatever.

403
01:09:48.000 --> 01:10:02.000
So is that, is it a decline in likes? Is it actually a decline in the relevance or the acceptance of a certain type of message or is it because the scale has changed?

404
01:10:02.000 --> 01:10:08.000
This is something that has to be dealt with when this is what they call the platform and what a for instance error.

405
01:10:08.000 --> 01:10:31.000
Something we haven't looked in. Until now platform coverage is what I also already talked about is the different inclusion probability So you know, Twitter uses for example in Germany at the age distribution of of Twitter users is very different from the general population and also are again different from the population that is using social media in general.

406
01:10:31.000 --> 01:10:38.000
So this is something you have to account for. Everything in orange, that the trace, the traces is basically the thing you collect, from these, from these platforms.

407
01:10:38.000 --> 01:10:53.000
So. The trace selection error is, refers to issues. Are you using the right or all the key words?

408
01:10:53.000 --> 01:11:11.000
For example, we used, Not a lot of synonyms or something that we got from the literature, but there are of course other means to do so so some people are using machine learning algorithms where the algorithm is learning how, what kind of keywords are used.

409
01:11:11.000 --> 01:11:20.000
We didn't apply this, but we referred, consulted the literature and did some own research about the the relevant search terms.

410
01:11:20.000 --> 01:11:31.000
Then trace reduction refers to, how do you get, yeah, which Which kind of, posts, for example, do you, do you select?

411
01:11:31.000 --> 01:11:45.000
So for example, when you look at telegram text posts And we. The problem is when you only look at text posts, you are and you are eliminating everything that is not a text post.

412
01:11:45.000 --> 01:11:55.000
Some, for example, especially on these telegram chat groups, they post a lot of memes or links or videos, but they can be seen as a statement.

413
01:11:55.000 --> 01:12:20.000
So what we did in the end, we were just counting the date variable as activity on the channel, which of course doesn't allow a high level of abstraction but at least we can cover a longer time frame and see maybe if the activity in a certain channel you can be seen as a as an acceptance of example.

414
01:12:20.000 --> 01:12:21.000
I'm sorry, have you have about 3 min left? So, yeah.

415
01:12:21.000 --> 01:12:28.000
Conspiracy narratives or something. So, 3 min left. Okay. Okay, okay, well.

416
01:12:28.000 --> 01:12:43.000
Then there are other selection errors and reduction errors. I will just go on briefly. We used, several, classifiers to identify political parties or journalists, which I used, I used the, the list from, Wikipedia.

417
01:12:43.000 --> 01:12:49.000
So, that can occur some, errors as well because II use the description and the links.

418
01:12:49.000 --> 01:12:59.000
Of the Twitter of the Twitter account. So, It might be possible that I identified some people who are not a political party or media party.

419
01:12:59.000 --> 01:13:04.000
Okay, I will show you here the Twitter search terms. In the end, we only have the Corona related one.

420
01:13:04.000 --> 01:13:16.000
So everything we looked at are we used the the ones on the right side within the one on the left side because the Twitter API was breaking down.

421
01:13:16.000 --> 01:13:29.000
I can show you brief descriptive, brief descriptive, results. So here we see the different topics, conspiracy, death tolls, attitudes towards masks and measures and the side effects.

422
01:13:29.000 --> 01:13:47.000
We see some peaks during the first lockdown when they call, talking about measures. And, you will see that conspiracy narratives, they peaked also around the first lockdown but got a little bit weaker and the development of the pandemic.

423
01:13:47.000 --> 01:13:49.000
Okay.

424
01:13:49.000 --> 01:14:01.000
Hey, we, we, we use the Raw negative score. It's which a sentiment analysis, what we can see, individual media and political party accounts, which I flag with my algorithm.

425
01:14:01.000 --> 01:14:17.000
We see that the there is a higher variance a higher amplitude of politicians or political political organizations compared to media for example the green line it doesn't have to media for example the green line it doesn't have the green line it doesn't it doesn't have this high amplitudes.

426
01:14:17.000 --> 01:14:28.000
And we see the post activity of selected telegram channels. So we have some, you know, our version of Alex Jones and, Qanon and so on.

427
01:14:28.000 --> 01:14:43.000
We see for example our organization that was mostly concerned with organizing against the pandemic matters the violet line they they start very slowly in the beginning but they get stronger over the overtime.

428
01:14:43.000 --> 01:14:51.000
And this is our, our, And infection activity data, but we hadn't still linked all of this yet.

429
01:14:51.000 --> 01:15:04.000
So which are the next steps of course the fusion of these of these data the harmonization of scales do we use that standard standardized scores or do we use more complicated algorithms?

430
01:15:04.000 --> 01:15:09.000
We have regional data. We didn't control for this yet. So we have the pandemic activity of certain regions.

431
01:15:09.000 --> 01:15:23.000
We have the attitudes of certain regions. We have the tweets of certain regions. Do we we didn't validate the data yet for example there is a Geezers panel on on tweets and we try to validate them with our data we have.

432
01:15:23.000 --> 01:15:38.000
We are thinking about using content analysis. of using, for example, certain posts or tweets and do some mixed methods analysis with this and we have looked at what kind of statistical model.

433
01:15:38.000 --> 01:15:51.000
We gotta use this at a cross lag panel or is it a, contemporary effect and so on, or a growth curve analysis we, still have to.

434
01:15:51.000 --> 01:15:57.000
Produce the complete final data set. You know, it's the first year of a 3 year old project.

435
01:15:57.000 --> 01:16:08.000
So, we will keep you up to date, when we have more progress. Thank you.

436
01:16:08.000 --> 01:16:16.000
Thank you very much and now we move to our last presenter who is here in person with us.

437
01:16:16.000 --> 01:16:21.000
Hmm.

438
01:16:21.000 --> 01:16:25.000
Yeah.

439
01:16:25.000 --> 01:16:29.000
I want the, share screen.

440
01:16:29.000 --> 01:16:38.000
Should be. And that's what I'm expecting. So I want to like that and share that.

441
01:16:38.000 --> 01:16:46.000
It works. And toss on my knee to adjust the laptop.

442
01:16:46.000 --> 01:16:53.000
Make sure. We see. Where can I check it? Yes.

443
01:16:53.000 --> 01:16:58.000
Oh, I'm checking it. Sorry.

444
01:16:58.000 --> 01:17:06.000
Thank you for having me. My name is Erek Saddowski. Main Institute of Political Studies of the Polish Academy of Sciences.

445
01:17:06.000 --> 01:17:20.000
What I would like to show you today is the case study. And one of the issue listed in the title of this session that is Comparability.

446
01:17:20.000 --> 01:17:41.000
In the context of. International service and one I would say every scholar in international comparative Sorry research has to deal with certain dilemmas.

447
01:17:41.000 --> 01:18:11.000
That involve measurement a equivalence. And this dilemma in contemporary context is well embodied in the debate taking place in recent years in several scholarly journals which involves a fiery exchange of arguments, which involves a fiery exchange of arguments between what involves a fiery exchange of arguments between i would stentatively call

448
01:18:17.000 --> 01:18:31.000
it the invariance party and complementarity or nomological parties. While the Methodological issue is all there itself.

449
01:18:31.000 --> 01:19:01.000
This debate started back in 2,015 when Aleman and Woods decided to review the analysis based on world value value survey and they came to the conclusion that most of these Analysis of most of the conclusions are actually incorrect because of the lack of actual comparability in terms of measurement in variants.

450
01:19:08.000 --> 01:19:23.000
The refugees themselves, Valsal and Inglowheart responded with the text that argued that there are several logics behind international comparisons.

451
01:19:23.000 --> 01:19:51.000
And, invariance that is focused. On item convergence is based on dimensional logic while they are using rather the combinatori logic which is focused more on external validity and predictive power of indicators.

452
01:19:51.000 --> 01:20:21.000
More than internal coherence of items. Later on, Outters. In other other teams of outer started to argue on some middle ground in this dispute so they advocated for a relaxing of a criteria of equivalence.

453
01:20:23.000 --> 01:20:36.000
But they also mostly advocated for the invariants. For the checking on invariance in

454
01:20:36.000 --> 01:21:06.000
Some not using some some statistical approach. But. Recently the punches started flying once again when Walsall and his co-autors generally argued that the measurement invariance approach is a kind of methodological dogmatism and called it and they called it actual Malpractice.

455
01:21:10.000 --> 01:21:27.000
They got the response from MERLA MAN and, and,ters who Refuted those claims and they themselves tried to point out that the arguments used by by Valsal and and his colleagues.

456
01:21:27.000 --> 01:21:57.000
Were incorrect. Finally this year, Robbie and Litka published a paper which is mostly in line with the argument by Nelson and England hard because It tells that in most cases internationally strict Scholar invariance is inativable or can be achieved in relatively few cases limited to Western democracy.

457
01:22:03.000 --> 01:22:24.000
And using the Partial invariance measures actually involves some arbitrary decisions so this is not a fully valid approach as well and they argued that we should focus on some other types of equivalence.

458
01:22:24.000 --> 01:22:39.000
In fact, they say that equivalence is not the same as invariants because there can be invariants without equivalence and and equivalence without without measurement in variance.

459
01:22:39.000 --> 01:22:53.000
So. If you Deal with the international comparative analysis. You have to deal with this dilemma somehow.

460
01:22:53.000 --> 01:23:23.000
You have to take into account the arguments put forward by both. Sites and decide on your own How to approach those arguments what I want to show you is my own dilemma that involved analysis of convergence in individualism or so-called idiosyncratic orientations.

461
01:23:23.000 --> 01:23:45.000
In other words, self-orientations across German and Polish birth cohorts. The study was based on on the one hand the German socioeconomical panel from which I have extracted data on selected births at different time.

462
01:23:45.000 --> 01:23:57.000
Points. This was a locked comparison. But the police data was more or less on the same cohorts.

463
01:23:57.000 --> 01:24:12.000
One of them approached adulthood at the moment when the communist fell in Central and Eastern Europe and another covert was born at that time.

464
01:24:12.000 --> 01:24:26.000
The first covert was surveyed twice in both cases and the Polish surveys use Mostly replicated questions from the German service.

465
01:24:26.000 --> 01:24:41.000
So in terms of equivalence of the questions. The language equivalence one would argue that this equivalence was achieved here.

466
01:24:41.000 --> 01:24:51.000
The question in question so to say was about key values things that are important in life more important, less important or not important at all.

467
01:24:51.000 --> 01:25:16.000
Among those 7 items that I show you that I that I have used in this analysis first, third and fourth are indicators of self orientations, can be interpreted as indicating allcentric orientations or orientations toward the others.

468
01:25:16.000 --> 01:25:35.000
Within Germany I'll also wanted to Compare Western and Eastern regions because this comparison, formed a kind of triangle with, former GDR.

469
01:25:35.000 --> 01:25:48.000
Communist. Eastern part of Germany, hypothetically was falling between those 2 cases. Now, there are several issues in this analysis that I had to deal with.

470
01:25:48.000 --> 01:26:08.000
First one. Involved systematic differences in that in data that that I have discovered

471
01:26:08.000 --> 01:26:19.000
First of all, proposed respondents in different surveys at different time points and those surveys were conducted by different survey agencies.

472
01:26:19.000 --> 01:26:35.000
They showed a larger propensity to rate all the values as more important. So basically the idiocentric and other centric ones were more important to them.

473
01:26:35.000 --> 01:27:05.000
Here in the table free you see the comparison of the, of the mean value. In all cases, police respondents were statistically, they statistically higher value valued every item on the on the list.

474
01:27:05.000 --> 01:27:18.000
So we had the systematic error which can be easily linked with the yes Iink effect or so so-called acquiescence bias.

475
01:27:18.000 --> 01:27:44.000
It can be speculated that it can be speculated that it can be explained in terms of cultural differences or perhaps some small methodological differences because The question on values was asked a bit earlier in a German questioner than in the Polish one.

476
01:27:44.000 --> 01:27:58.000
Mmm, the second systematic difference. Involved how those items scaled up in exploratory factor analysis.

477
01:27:58.000 --> 01:28:28.000
The Allacentic and Idiocentric items did not form one vector of individualism we instead rather observed orthogonal factors and what is very interesting is that the the allocentric A lot and scale was not consistent between Poland and Germany, but was also not consistent within Germany.

478
01:28:31.000 --> 01:28:43.000
So if you wanted to compare Eastern and Western regions. You would be you wouldn't be able to achieve Full colorar invariants.

479
01:28:43.000 --> 01:29:06.000
This is interesting because the scholar invariance criteria. Makes it hard to do completely valid international comparisons but in fact if you look into regional differences then some surveys don't achieve this equivalence within countries.

480
01:29:06.000 --> 01:29:30.000
Like in this very important social economical, panel study. So II had to somehow deal with this dilemma one of the recommendations is to skip some of the items that don't scale very well.

481
01:29:30.000 --> 01:29:59.000
So I ended up analyzing only the idiocentric ones. As you can see in table 5 The relationships between items were fairly similar in Poland and Germany are the only difference is on the correlation between item one Item on affording on material values and item for importance of career.

482
01:29:59.000 --> 01:30:21.000
It was a bit higher in Germany than in Poland. But another thing that I've, I have done before employing full analysis was centering of the items.

483
01:30:21.000 --> 01:30:35.000
On the grant mean. In this served the purpose of having of getting rid of this systematic bias.

484
01:30:35.000 --> 01:30:44.000
Towards putting more importance in all the values within Polish. No, we didn't polish samples.

485
01:30:44.000 --> 01:31:07.000
Finally, by reducing number of items and Bye Centric I was able to achieve not this full scholarly variance but at least measurement and geometric variance as I could constrain the factor.

486
01:31:07.000 --> 01:31:22.000
On on items in the exploratory factor analysis and that finally met the criteria for at least this partial kind of Invariance.

487
01:31:22.000 --> 01:31:52.000
Now, it led to another dilemma in fact and another interesting issue because while I have found mostly consistent patterns that showed a reduction in individualism in men both in Poland, Eastern Western Germany, there was no such a drop between significant control between generations.

488
01:31:56.000 --> 01:32:24.000
In women although we see some age effect There was one prominent outlier. It was for younger cohort in eastern Germany because there we could see the stronger.

489
01:32:24.000 --> 01:32:48.000
Joint cover and period effect done the joint H and and period effect. For the comparison I decided to make this analysis once again in a very crude manner so I've simply computed the index of scores summing up the answers and then putting this in the linear regression.

490
01:32:48.000 --> 01:33:17.000
And what is interesting in this very basic analysis. I have achieved, I would say, a full conference of, the patterns so that outliers simply banished with this simpler solution to the comp.

491
01:33:17.000 --> 01:33:46.000
So I now move to the final issue which or the final questions which actually I'm going to leave you with The question is should we pursue The methodological rigor or rather consistency of Res the consistent pattern that we see in in the final results of our analysis.

492
01:33:46.000 --> 01:33:59.000
And this is basically that dilemma that I have started with between the 2 a logics in comparative Analysis. Thanks.

493
01:33:59.000 --> 01:34:04.000
Thanks for for being here. Thanks for