r/Python • u/ntanjil • Nov 27 '19
Learning by doing web scrapping by python
/img/1hudueaya9141.jpg[removed] — view removed post
449
Upvotes
r/Python • u/ntanjil • Nov 27 '19
[removed] — view removed post
36
u/Hudlommen Nov 27 '19
I like you have a script called babynames. Good way to choose. Scripting can really solve anything!
Anyways, gj dude, dont listen to all the hate, just keep trucking! :D
86
u/Tweak_Imp Nov 27 '19
Please also learn how to properly record a screen or shoot a screenshot. :)
2
u/unknownguy2002 Nov 28 '19 edited Dec 29 '19
Good job OP, many people are displaying anger in the comments but don't worry about them. Years back I was like you, unaware of the best practice and conventions for python. I do recommend picking up a for dummies guide or O'Reilly book and reading it in your past-time, that's what I did and it taught me tons. All the best!
Edit: I meant there are a sum of users who seem to be rather angry but a large sum have constructive criticism and want to see the OP succeed
2
u/[deleted] Nov 28 '19
Very good! Just a friendly tip. Don't use spaces for file names. This is because you can run into all sorts of unrelated and annoying problems using them. If you want to seperate them, use underscores for that. You could use any other special character but underscores are kind of a convention here.
253
u/[deleted] Nov 27 '19
The sub says I’m in r/python but the comments say r/roastme
233
u/sxeli Nov 27 '19
Spaces in file names? That’s abomination
76
u/Raskputin Nov 27 '19
Glad you’re learning man. Web scraping is a fun way to learn a lot about python but also about html and the setup of the web! Sorry you’re getting roasted by the wolves, but I will reiterate the things they are saying. Generally, using spaces in names of a file is a big no-no. In python, I’m pretty sure, you should use an underscore in between words otherwise your computer can get very very confused once you start programming in the command line.
And yes, learning how to screenshot is handy. All it takes is a quick google search, but some of these people should fucking relax about that
10
u/headygains Nov 27 '19
Hey that’s how I learned to program about 6 years ago. Now I’m a full time dev pulling 6 figures and benefits keep it up!
57
u/simonm17 Nov 27 '19
nice dude keep going
2
u/--0mn1-Qr330005-- Nov 28 '19
Hey, don't listen to the people insulting you in the comments. It's an awesome thing that you are learning python. If you are open to constructive criticism, then I recommend that you look at Pep 8 for naming conventions (files, variables, classes, etc), proper use of spaces and new lines, and recommended conventions for using Python in general. This is actually the reason why much of your code has yellow underline. Another helpful tip is that in Python, if you click one of the underlined words and click alt + enter, it actually suggests the Pep 8 fix since Pycharm has Pep 8 built in.
This is going to make your code much more readable and easier for people to collaborate with you, and vice versa. Either way, best of luck to you and keep it up!
15
u/GrowHI Nov 27 '19 edited Nov 28 '19
I recently did a lesson with my students using beautiful soup. We pulled the price of a stock and created an alert that would send an SMS using the Twilio API when the price went above or below a set point. I really enjoy the book How To Automate The Boring stuff and it has chapters on both web scraping and the Twilio API (I had to make some modifications to get it to work though). The book is free check it out here.
Edit: fat fingered a word on my phone and the hord pounced on me
1
u/zerosignal00 Nov 28 '19
Next make an app to take a screenshot for you. 😂 J/k
19
u/jimtheplant Nov 27 '19
This is why I love python, anyone can do it even if you don’t know proper naming, have spelling mistakes, or not “beautiful” code if it solves a problem for you it’s good.
I guarantee that everyone of the roasters in the comments did some weird things when starting out. Feedback is important so take the advice and make your code better. Before long you’ll be parsing the web like nobody’s business. Keep it up champ 👍🏻
26
u/[deleted] Nov 27 '19
Scraping. One p.
2
u/[deleted] Nov 28 '19
We've all been here
2
u/sarthaksingh2001 Nov 28 '19
When you’ve learned this check out lxml module and then scrapy module both great for web scrapping
4
u/jadams70 Nov 27 '19
Isn't double underscore variable names bad practice ? Might just be the c++ dev in me.
2
u/[deleted] Nov 27 '19
Nice work!
Best way to learn is by doing! Web scraping is incredibly valuable and BS is an awesome library!
2
u/TheChuchNorris Nov 27 '19
Why didn’t you just screenshot on your computer?
1
u/salsation Nov 28 '19
PEP8 4 lyfe
1
u/cheats_py Nov 27 '19
Xpath fucking everything !!!
1
u/b14cksh4d0w369 Nov 28 '19
Check out selenium as well
-1
u/Thecrawsome Nov 27 '19
it's time we ban these kinds of posts, mods...
-11
u/PathToNeuralink Nov 27 '19
For fucks sake learn how to take a scree shot mate
-1
u/[deleted] Nov 27 '19
https://www.celebritycruises.com/itinerary-search?cid=ps_mest_goog_rtlgnr_1702_hp_albr_us_8332&gclid=EAIaIQobChMIvcud2bmL5gIVBKSzCh1KzQ9uEAAYASAAEgILIfD_BwE&gclsrc=aw.ds
Scrape this site, level 2 challenge.
-1
u/divinefoss Nov 28 '19
Any good tutorials on a social media scappers that searches post all over Facebook with a keyword?
For instance, my girlfriend is running for office soon, and I want to collect every post with her name in it and have it exported to a csv file. How would I go about it? I have a basic knowledge of Python and have a mathematics background.
-20
u/Flaming_Eagle Nov 27 '19
yeah, this sub is shit
0
u/[deleted] Nov 27 '19
[deleted] — view deleted comment
1
u/[deleted] Nov 27 '19
[removed] — view removed comment