Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Google Sheets can be surprisingly handy as a UI for editing data. For example, I have a job on AWS that scrapes a handful of URLs and snapshots them. Instead of creating a database and hosting it somewhere, or hard-coding them, I put them in a spreadsheet that the AWS job reads at the beginning of every run.


I just made an example for how to do this using Refinery (most of the pieces were off the shelf using "saved blocks").

https://app.refinery.io/import?q=9lvohdvqd50g

This project will:

- Run every hour (via timer)

- Grab URLs from a Google Sheet

- Go screenshot all of the URLs

- Write each screenshot to an S3 bucket

- Email you at the end after the jobs finish

There is more I could do with this (like trigger an email if something fails, etc). But at least does what you describe above in a (hopefully) succinct fashion!

Also, shill alert as I'm an author for this service (but this was a fun project to build over a few minutes, so thank you).


Aren't you forced to use Google Apps Script to interact with Sheets? I remember being forced to use it for interacting with Google Forms API.


> Aren't you forced to use Google Apps Script to interact with Sheets?

No, it has a REST API; you need Apps Script if you want automation within Sheets, but you can interact with Sheets without it.


That's the best part, I don't even use an API! I just go File -> Publish to the web, then select "Comma-separated values" instead of "Web page" from the dropdown. Then I use Pandas to read the CSV in Python.


I continually find myself amazed with what people can do with Excel.


haha yeah and I always wonder why especially engineers are so creative in using Excel as a graphics/painting tool :) Looks good, but a nightmare to read in a programmatic way ^^


How do you get around the number of fetch limitations?


I'm hitting at a rate of 2/hour, it hasn't been an issue.


how are you accomplishing that? Does Google sheets have a scheduler built into it? Long running timer?


Google Apps Script has a scheduler built in: https://developers.google.com/apps-script/reference/script/c...

I use this to automate a lot of tasks. E.g. automatically clip supermarket coupons every week, check for tax loss harvesting opportunities in my portfolio, etc.

It's really powerful


The scheduled job runs on AWS, it just fetches the sheet when it runs.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: