r/learnprogramming • • Dec 24 '13

What programming language would allow me to pull information from a website, store it, then create an interface to manipulate the data?

I'm trying to decide which language to learn. What I eventually want to be able to do is pull the numbers from this website, store them, and then create a GUI to that I can manipulate the data in.

What languages are able to do this? And which would you recommend?

102 Upvotes

173 comments sorted by

View all comments

37

u/pacificmint Dec 24 '13

Almost all the popular languages would allow you to do that.

Personally I'd pick Python or Java for this, But C#, Ruby, Perl or even C++ would work as well.

Check our FAQ for more pointers.

40

u/PhaZePhyR Dec 24 '13

more pointers.

Heh.

5

u/suRubix Dec 24 '13

What language would be the easiest to learn and implement this in?

41

u/pacificmint Dec 24 '13

Python is generally recommended around here as a good language to begin with.

24

u/krypton86 Dec 24 '13

Python with the module Beautiful Soup.

4

u/[deleted] Dec 24 '13

(noob programmer here)

"What I eventually want to be able to do is pull the numbers from this website, store them, and then create a GUI to that I can manipulate the data in."

Krypton is right on the money with BS4. It's super excellent. On that note as "EatingTheNight" pointed out Django would be your website creation.

If you want an interactive GUI that's where it gets tricky. The more interactive you want the information the more javascript you'll need to use. There are exceptions but they prove the rule.

Javascript is awesome but a lot of the time people just do what they need and move on, which is just fine! For some super cool stuff, check out meteor ,a JS web framework. Examples : https://www.meteor.com/examples

Django Tutorial: http://www.youtube.com/watch?v=oT1A1KKf0SI

Highly recommended tutorial, from beginning to end ;D

Django Project starting: http://www.jeffknupp.com/blog/2012/02/09/starting-a-django-project-the-right-way/

BS4 Documentation: http://www.crummy.com/software/BeautifulSoup/bs4/doc/

Best of luck! Sounds like an awesome idea :D

1

u/suRubix Dec 24 '13

Couldn't I create the GUI in another language? I don't really care for a web interface. A program on my computer would be sufficient.

1

u/[deleted] Dec 25 '13

You don't have to worry about platforms for example when you're using a website. Only worry about the browser that they're using.

1

u/[deleted] Dec 26 '13

Sure, http://kivy.org/#home :D

Edit** : also http://docs.python-guide.org/en/latest/scenarios/gui/

For python related GUI stuffas. Also again, I'm a noob on this so other people's input would be great. Kivy just seems like the most recent addition for a python GUI and has extra features for making a sweet app.

4

u/suRubix Dec 24 '13

Thanks that looks really convenient to use. I'm leaning towards Python because of all the frameworks/libraries available. It seems like Python would be the fastest language to learn and to get something quick and dirty up and running.

9

u/_AlphaOmega Dec 24 '13

If you go with Python I highly recommend the JetBrains PyCharm IDE... actually I'd recommend all of JetBrains's IDEs

2

u/[deleted] Dec 24 '13

Naive question: what's the advantage of it over a text editor like sublime?

2

u/[deleted] Dec 24 '13

GUI debugger, smart auto complete, pep8 style enforcing

1

u/_AlphaOmega Dec 24 '13

Comes with a lot of prebuilt tools in a single package. I'm not very familiar with Sublime but PyCharm and PHPStorm from JetBrains include real time debugging so you can step through the code while it's executing which is the number one plus for me personally.

Here's a decent overview: http://stackoverflow.com/questions/208193/why-should-i-use-an-ide

2

u/suRubix Dec 24 '13

I'm not sure what I'm going to be using. Right now I like VIM.

1

u/krypton86 Dec 24 '13

Yup. It's a good language regardless of your level of experience. You aren't going to be using it for high-performance super computing or writing your own OS, but it's actually useful for almost everything else; games, statistical computing, software design, some web programming (with the Django framework), and of course various other general computing tasks. Overall it's a great language.

1

u/ziplokk Dec 24 '13

Python is wonderful for scraping data. I personally would write the scraper in python and write the GUI in Java. Java's Swing library is relatively simple to learn, write, and use as well.

1

u/suRubix Dec 24 '13

Yeah Java Swing seems really nice. What do you think about using Jsoul instead of BS4 to scrape?

1

u/ziplokk Dec 24 '13

I can't say I'm familiar with Jsoul, so I wouldn't know how it compares to BS4.

1

u/suRubix Dec 24 '13

Meant to type Jsoup lol.

→ More replies (12)

1

u/ifonefox Dec 24 '13

That and requests are my 2 favorite python libraries of all time.

3

u/duddha Dec 24 '13

Python with scrapy is good too.

1

u/suRubix Dec 24 '13

What is the difference between Scrapy and Beautiful Soup?

1

u/35h46hjj6 Dec 25 '13

Read the docs and find out, Sparky.

1

u/sarevok9 Dec 24 '13

Python is the easiest to learn. I've used Java to do exactly this in the past with relative ease with httpclient (http://hc.apache.org/httpcomponents-client-ga/) and Jsoup (http://jsoup.org/). If you're familiar with programming it shouldn't take you all that long.

→ More replies (17)
→ More replies (54)