Minebase: a free data mining tool for social networks
Congrats on shipping this! Personally, I wish that social networks themselves offered bulk export of public assets for academic purposes instead of having people re-invent the wheel every time.
Just a heads up, IANAL, but make sure that you're considering Twitter (and other social network) API terms of service:
> If you provide Content to third parties, including downloadable datasets of Content or an API that returns Content, you will only distribute or allow download of Tweet IDs, Direct Message IDs, and/or User IDs.
> You may, however, provide export via non-automated means (e.g., download of spreadsheets or PDF files, or use of a “save as” button) of up to 50,000 public Tweet Objects and/or User Objects per user of your Service, per day.
> Any Content provided to third parties remains subject to this Policy, and those third parties must agree to the Twitter Terms of Service, Privacy Policy, Developer Agreement, and Developer Policy before receiving such downloads.
> You may not distribute more than 1,500,000 Tweet IDs to any entity (inclusive of multiple individual users associated with a single entity) within any given 30 day period, without the express written permission of Twitter.
[0] https://dev.twitter.com/overview/terms/agreement-and-policy
undefined
I'm guessing you might get a cease and desist. Instagram just shutdown Instagress, MassPlanner and others. Granted this isn't a bot but scrapers usually aren't met with much love either. Especially if you start selling the data.
This is my first web app and, while rather simple, should provide some use to people needing to export large amounts of data from Twitter and Facebook. In the coming weeks I'll be adding some new exports and support for more social networks. I'm really excited to be able to start showing it off and can't wait to get some feedback. Thanks!
Has anyone noticed one of the site maintenance template?
https://web.archive.org/web/20170627020124/https://minebasea...
You can find it on
For anyone who's signing up now or asked to get an email notification when a report is complete, my ESP cut me off saying I'm having an unusual amount of requests. I'm contacting them to get it fixed, but be aware you might not get emails sent right now.
With all due respect, it is not really a "free tool", it is seemingly a site where you have to sign up to get something.
This is also not particularly reassuring (I understand that being a beta it is probably only a temporary page copied from somewhere on the web):
https://minebaseapp.com/privacy
>Credit card information
We do not store your credit card information directly. It is passed through to a third party credit card processing merchant using 128-bit SSL encryption. We only keep your credit card's brand, last 4 digits and expiration date to assist and remind you of any billing related issues.
...
>Last updated April 1st 2042
Same goes for "Terms of Service":
FYI your confirmation email uses https://www.minebaseapp.com (not working) whereas the main site uses https://minebaseapp.com (working). I guess you didn't configure your DNS records to handle the www subdomain because that results in a DNS error.
The follower export widget on the homepage has 'What number will be rolled?' as the title text for the user drop-down (`#user_count').
Is that the actual text that is meant to be there or was it mixed up?
Hello @hobbescotch, is it possible to describe the techniques used to scrape the data, what technology stack are you using, where are you hosting, what would be the premium features?
Craigslist put the kabash on something similar.
Why won't other sites?
Im a sociologist. You made my life a little bit easier. Thank you.
How isn't that breaking Facebook's TOS?
I'm having trouble connecting
Halifax, NS or Halifax, England?
undefined
Hi, what data does this output?