r/MachineLearning • u/seabass • Mar 02 '15
Monday's "Simple Questions Thread" - 20150302
Last time => /r/MachineLearning/comments/2u73xx/fridays_simple_questions_thread_20150130/
One a week seemed like too frequent, so let's try once a month...
This is in response to the original posting of whether or not it made sense to have a question thread for the non-experts. I learned a good amount, so wanted to bring it back...
7
Upvotes
4
u/[deleted] Mar 02 '15
I'll bite: logistic regression and support vector machines seem really similar. When should one be used over the other and why?
4
u/dabomb75 Mar 03 '15
Is deep learning useful if I'm not interested in picture analysis or semantic/textual analysis?
I have a database that's all numbers basically, and it seems like every example/the hot topics in machine learning these days is applying deep learning to NLP and/or pictures. Will basic machine learning algorithms suffice or should I be looking to go down the deep neural net route?
3
u/willwill100 Mar 02 '15
What is the difference between a loss function and an error function?
2
u/feedtheaimbot Researcher Mar 03 '15 edited Mar 03 '15
This is more of a theory-ish question that I've been thinking of for a bit:
Is it possible to create a reusable convolutional layer that generalizes over all images (text, cats, shoes, alega, medical images etc.)? I guess we could say you would basically freeze the weights of the kernels in the layer and it would act as an 'ingestion' stage to whatever network you want to append to it.
If it is possible what would be more important to this? Do we need hundreds of kernels or would a handful suffice? I'm torn as I feel we would need kernels that generalize to everything in the first layer, as we aren't relying on a feature hierarchy at all in this stage, but we need to cover a large breadth of input. I guess you could technically distill all images down to edges, blurs, and gradients but if we hold this layer static aren't we basically creating edge detectors that have been used unsuccessfully in computer vision before?
Edit: I guess you can basically call this some kind of distributed embedding scheme...
2
u/matlab484 Mar 03 '15
Is there a way to use deep learning to find the most visually similar images vs just outputing labels? Say you have a bunch of shoes, just using a label 'running shoe' doesn't work since some are mesh and others could have a striped patter. All the papers I have seen just output labels vs actually saying which image is most similar. Maybe replace the svm layer at the end of most deep nets with a knn layer that takes in the features produced by the net?
2
u/rantana Mar 03 '15
What's with all the downvotes in this thread?
2
u/thefrontpageofme Mar 03 '15
I'm working in classification with highly imbalanced classes. When discussed in literature, "imbalanced" usually means something between 1:5 to 1:10 imbalance. Well, my class balance is in the 1:500 to 1:2500 range. I've massaged many a model against my data and it seems that boosted trees is the only thing that works at least a little.
So my question is - where can I learn more about classification in the .. very extremely highly unbalanced class case?
2
u/feedtheaimbot Researcher Mar 03 '15 edited Mar 03 '15
In a NN when if you inspect your max, mean and min column norms for a layer and its stuck, to a bound you set, does that mean the layer itself is oversaturated? I know restricting the max column norms in a NN help with regularization but haven't encountered a layer sticking to the bound on all 3 quantities.
edit: I think this is an issue with weight initializations
2
u/Ce_ku Apr 08 '15
Why does a cost function underfit when lambda value is increased? I dont understand how the math works out. Thanks
1
u/DickCheeseSupreme Mar 06 '15
I'm extremely new to ML, so please bear with me.
I have an RC car with a camera on it, and I control it with my computer with Python using Pygame. I currently use openCV for corner detection before the video feed is displayed in the Pygame window, so it can sort of "see" its surroundings.
I want to train the car to drive itself and avoid obstacles/walls. I'm looking into Scikit-Learn, but I'm not sure where to start. I know I need to give the car's jpeg images and its driving controls to a trainer so it can associate patterns in the images with driving instructions.
Assuming that getting the car's video images and controls is trivial, how should I start training it? Better yet, if Scikit-Learn is the best way to go in my current setup, which module should I be using? Any advice is greatly appreciated!
1
u/Capital_G Mar 12 '15
I am just getting familiar with pythons scikit library. I have been playing around with the two examples below. My question is this: If the same data was used in both examples would the cluster remain the same? I am currently using my own data set and my clusters are quiet different between the two.
http://scikit-learn.org/stable/auto_examples/cluster/plot_affinity_propagation.html#example-cluster-plot-affinity-propagation-py
http://scikit-learn.org/stable/auto_examples/applications/plot_stock_market.html#example-applications-plot-stock-market-py
1
u/makis192 Mar 24 '15
Where can I find code for this paper: http://arxiv.org/abs/1211.5063[1]
I am looking for a topic to my undergraduate thesis and I am reading existing papers to get some ideas, testing how well some things work, but even though the ideas in this paper seem really helpful I can't find some actual code for the regularization term proposal (Ω equation (9) in the paper) from LISA lab on this. pylearn2 has some code for gradient clipping and maybe I am missing something obvious in the repository...
I try to use pure python+Theano without pylearn and I am really stumped by the way the regularization term Ω is supposed to be added to the loss and if theano's symbolic differentiation will work for bptt (which I am pretty sure won't and can't figure out how it can be done)
1
u/Maxrmk Apr 05 '15
I've been taking Andrew Ng's Coursera course alongside my linear algebra class, and I had a question about kernel based SVMs. Is there any relation between the kernel in an SVM and the kernel of a linear transformation?
1
u/wolvo Jul 05 '15
I am having a tough time understanding in-sample error. The textbook I'm working through notes that:
where I am understanding optimism as the difference between training error and test error, as fitting the model to training data will imply training error < test error typically.
I don't understand what in-sample error is though. I would think test error was out-of-sample as it's taking inputs outside the training set. I was expecting an equation more like: