November 12, 2020
If you want to get more responses for your survey, you can always check out /r/samplesize on Reddit, where of the respondents are in the younger demographics (18-34), mainly from English-speaking countries. The political lean of the site is generally, but not exclusively, to the left.
How to get the most out of the subreddit is a bit less obvious. Namely, there are a few rules that you need to follow when posting:
[Academic], with one of those respective tags in the title
[Repost]tag in the title
For example, you could have a survey titled
[Academic] How much do you sleep per day (Anyone 18+), or
[Repost][Marketing] What kind of fish do you eat? (anyone living in US).
There are additional rules for posting, but these are the ones to keep in mind when setting a strategy for posting.
You also need to create a Reddit account if you haven't already. All you need is an email address, and it's free!
Usually I don't get more than a few dozen responses, although in a few cases I've gotten several hundred. The most important factor in getting responses is getting more upvotes, thus a higher score, on your submission.
The Reddit algorithm will push newer posts up closer to the top of people's pages, as well as posts with higher scores. The process is a positive feedback loop: The higher score you have, the more likely people are to see your post and upvote it and/or fill it out.
There is the question of what the best way to post your survey is. Here are a few guidelines + observations, and then I will back up these claims with data:
?v=1to the end of the URL, or any number. If there's already a
?near the end of the URL, you can add
&v=1(or any number) to the end of it instead..
Below is a graph of different posting times and the percentage of submissions with scores over 10. Tap on the image to show more technical details in each cell.
The best times to post are generally in the morning for the US, around 12:00-15:00 UTC (or 7:00-10:00 am EST) Saturday
Additionally, in an earlier analysis for all of Reddit, that time range was among the best for posting, although Saturday and Sunday morning in the US seems to be the best overall, with a wider time window for posting (up to 6 hours earlier and 3 hours later). Thursday does not appear to be a very good day for posting.
See the limitations section at bottom of page for why the above might not be a true cause-effect relationship.
Although planning when to do the survey is important, one can alternately look at factors that improve the average score of a post. The below graph highlights the expected percent increase in score based on the following factors.
The main takeaways:
I used my Tree Grabfor Reddit Scraper (GitHub link) to collect the most recent posts on /r/samplesize, and then for each commenter/poster, look through their past 1,000 submissions to see which ones were submitted to /r/samplesize. Posts from April 2020 and onwards were used, and must be at least 19 hours and 12 minutes old as of scraping (0.8 days). The number is a bit arbitrary, but most posts usually reach their maximum potential by that time.
7,533 posts from /r/samplesize were used in this analysis.
In no particular order, here are some possible caveats to the above analysis:
Additionally, there appears to be a higher proportion of casual posts on the weekends, and a lower proportion of marketing and academic posts. See the below table for proportion of posts by day of week:
For the effects of different post characteristics, I used a linear regression model using bidirectional stepwise selection (w/BIC). The response variable was
log(1+score), which doesn't perfectly correspond to number of votes on a log score, but it is close enough for general interpretation.
Code and data used for the analysis can be found on https://github.com/mcandocia/samplesize_regression.