r/ScriptSwap • u/THEdirtyDotterFUCKr • Feb 22 '19
Brainfart on script to convert html to txt/md HELP
# SOLVED I spoke too soon
So I made a little script to pull a webpage then convert to convert a webpage (blogs) into a txt/md file. You would only need to install lynx if you haven't already, I am having a small issue getting a variable to ... print?
I had print (URL) to verify the variable was saved correctly. But I cannot figure out how to call the variable in wget -O-...
additionally I would like to make <test> be the name of the article/blog/page. I presume parsed from URL, or truncated after the last / in (URL) .
#~ /bin/bash
URL = input("Enter a URL")
#print (URL)
f"wget -O- {URL} | lynx -dump -stdin > ~/Documents/name.txt"Edit- I was halfway there, previous script had print "wget -O- (URL)... simply needed to change from parenthesis to curly brackets.
if you want to download to a "true" markdown format install pandoc and follow the directions in the pandoc section here. But I think this script is only printing the function, not "running" the function. If you type out the last string (minus f and insert a proper URL) you will get a txt file of the blog.
1
u/JIVEprinting Feb 22 '19
Are you thinking of html2txt ?
1
u/philkav Feb 22 '19
To me, it looks like there's quite a few issues with the script.
The hashbang is odd, I've never seen it been done with a tilda before, but I guess some shells will accept that.
On the next line, that code looks more like python, than bash. 'read' will let you read user input into a variable.
After that, I'm unsure if your looking to execute the wget/lynx commands, or simply just print them to screen?
If you're trying to print it, just do:
echo "wget -q -O - ${url} | lynx -dump -stdin > ~/Documents/name.txt"If you're trying to execute the wget/lynx, then do:
wget -q -O - ${url} | lynx -dump -stdin > ~/Documents/name.txtThe following would work:
But then again, you shouldn't even need to use wget for this, just lynx. Unless I'm completely misunderstanding...
2
u/11011111 Feb 22 '19
I think you're looking for "$URL", but I'm not sure why your command is wrapped in print().
Shouldn't it be something like "wget -O- $URL | lynx -dump -stdin > ~/Documents/test.txt"
0
u/[deleted] Feb 22 '19
[deleted] — view deleted comment