Deep Learning is currently an empirical science guided by intuition of practitioners. A main principle in experimental sciences is that a theory without predictive power is not considered a full-fledged theory. As such, unless they are interesting predictions coming from their theory (rather than only barely justifying existing empirically observed phenomena), this is just speculative theory that I would not use the phrase "Foundations Built" for.
As an example of this general litmus test for a theory see e.g. Eddington's confirmation of GR: https://en.wikipedia.org/wiki/Tests_of_general_relativity#De... . If there are hitherto unknown phenomena in DL predicted by this theory then I'd stand corrected and concede that there may be something to these theories.
Yeah, it's not foundations like elementary or axiomatic.
An actual theory of ANN and then of NNN will be of more momentous import than of any previous phenomena, will usher in a fundamentally transformative age where we understand ourselves, and will surely require a fundamental breakthrough in pure mathematics... probably a great many.
The article seems pretty upfront about the fact that we are nowhere near a solid theory. One of many quotes to that effect: “ […] beginning to build the rudiments of a theory of neural networks.”
The main point seems to be why such a theory is highly desirable. To that end, it does a far better job than the somehat tired Feynman aphorism about good theories “giving us more than we give them”. Namely that the current state of the art of “intuition” and vast trial-and-error runs being somewhat embarrassing for a field so closely related to math and statistics.
What people make is ultimately what demand & supply dictate. On the other hand, for one-off cases where the market is super small, people make what they can negotiate. Of course, even in that one-off cases there is data about the pay of superstar within previous movies/shows/engineering orgs to use as an anchor.
But what about your analysis which is based on unit economics. How does that tie in? (because it is of course relevant) By the fact that if you hire lots of super stars by market price, but your unit economic does not allow the operation to be profitable, you will eventually have to shut the operation down. Good bye super stars and whole operation!
Except for Elon Musk, who with all due respect, I believe does not count as an informed person in DL.
It really bothers me that he (who is clearly an excellent entrepreneur and human being in general) makes such strong statement on a topic he is clearly not an expert in and is so vocal about it too. Given the amount of influence and number of online followers he has, I find this irresponsible.
Elon's training is in physics and economics. He understands linear algebra and programming perfectly well. He also understands that policy (economic or otherwise) won't be developed unless people put it on the government's plate. And he understands that government tends to be reactive - they are invariably way behind the curve in governing tech. So he's using his bully pulpit to get the troglodytes (a concept that maps nicely to many Republicans) and luddites (a concept that maps nicely to a lot of Democrats) in Congress to start working on the problem (for context, Ted Cruz of all people runs the Senate science and technology committee).
Yup. Machine learning driving the optimization loop, with humans working as smart sensors and actuators, responding to high-level commands. That's a pretty powerful AI right there.
Did he really say that specifically about the "current trend of deep learning using neural networks"? Surely he would have been speaking more generally about AI?
See the references I give below; if he is talking about general trends in AI and is giving an estimate of 7 or 8 years for emergence of AGI, where is he basing it on? Is he basing the estimate on unseen breakthroughs (i.e. unknown unknowns) to occur within that 7 or 8 years frame, or is he talking about current DL techniques pushed to their limits? For the former, nobody can make a reasonable estimate (nature of unknown unknowns) and for the latter, most experts in the field seem to agree that that current techniques do not lead to AGI. (Like have you seen how hard it is do things like visual question answering (VQA) or text summarization tasks? These are much much simpler than AGI but like AGI do not lend themselves easily to supervised learning.)
He talks about AI in those links, and does not mention specific approaches like DL etc. He's been around long enough to see the different approaches to AI, so as to not confuse it with a specific technology. I doubt it would even occur to him think AI=DL.
Many people have made estimates for human level AI, without relying on specific technologies. e.g. Vernor Vinge, Ray Kurzweil, Bill Joy. For example, the lower part of Vinge's range included the 2020's - for his prediction made a few decades ago.
Is Musk accurate? How can any of these be very accuracte, with so much unknown, as you note. Is he optimistic (er, predicting too early)? I guess so.
I'll note that even for Musk's own companies, upon which he is the expert, and upon which he wields more influence than anyone, he is notably optimisitic in his estimates...
Anyway, the answer to my question is: no, he didn't speak of those specific technologies, just AI.
I assumed akvadrako downvoted that since the vote and the comment appeared very close to each other and both right after my comment. But of course, I surely could be mistaken.
And anyways, downvoting is not a big deal and they have the absolute right to do so. It's just that stating why you disagreed or downvoted would be helpful.
I don't know if it's still true, but it used to be the case that if your reply to a comment your downvote to that comment does not count. It was a pretty clever mechanism in that you could be certain any downvotes didn't come from people replying to you.
I don’t see why HN doesn’t require every down vote to have a comment. Judging from the sheets of gray on most threads, most down voted comments aren’t actually in violation of site policy, off topic, trolling, etc. It’s just a few little people disagreed. Which is fine! But it’s not constructive or interesting to just slap the commenter with your dick.
Because then every other comment would have a stream of replies justifying the downvotes - replies which, presumably, would be regular comments, and thus susceptible to further downvotes (with comments)!
In other words, this would be a huge noise generator.
Very good points; I'm in general though space skeptic in the following sense. Human race as a whole always seeks more, bigger, further, faster as solution its problems. I don't think space is the solution to our earthly problems.
Maybe 1000 years from now, people look at us and say god damn it. I wish they had stayed there and learned how to live in earth well. We are now speared all over the place and we all hate hate each other; and just right now I hear Martian went ahead and annexed Venus and they also seem to be interfering in Andromeda elections.
AirBnB is such a great business. No significant competition for now among startups. Primary (initial) market is different from the incumbents (hotels) and it can gradually chip away from their market share too.
It also has better network-effect than say Uber (which has per-city network effect but no significant global network-effect) and less competition than latter. The "moat" is super strong and the leadership (Brian, Joe et al.) seem to be really good.
I'm so in awe of their prospect. My only misgiving is that they rejected my application (which proves no one's perfect after all - as they seem to also make hiring mistakes:P)
Is VRBO/HomeAway not a significant competitor? It's owned by Expedia, so it has a pretty huge backing.
I usually check both AirBnB and HomeAway when I'm looking for a rental and don't see any reason to be loyal to one over the other. (AirBnB has a better website/app, but the whole experience is pretty similar.)
AirBNB has their own version of a 'showrooming' problem as well - last time I went to Hawaii, I found a nice condo that I liked, then searched for that same condo online, found the owner's standalone website offering similar prices but no AirBNB fee. I booked direct.
That's a pretty serious amount of effort to go through, and I don't think is actually a meaningful problem.
Most hosts do not have an alternate presence and it should be pretty easy to contractually forbid this behaviour on the part of hosts, at which point it's a detection problem, rather than a fundamental problem.
Exclusivity contracts? What's my incentive as a host there? Is AirBnB going to give me more of a cut?
Is AirBnb going to make "exclusivity" actually mean something, and not give me 100s of hits in a square mile or two, so that my listing doesn't appear on page 10 of the results?
It's possible I've misread the host pool, but 80%+ of the places I've stayed at have not been professionally managed apartments, but rather people subletting a room or their entire place when traveling, and doing some quick reverse image searching shows that these places are not available on any alternative sites, for them an exclusivity arrangement doesn't even matter because they were not going to rent elsewhere.
If people sidestepping AirBnB became common, it would be a pretty easy decision for AirBnB to enforce exclusivity and just say goodbye to everyone else, because people showrooming on AirBnB is actively harmful to them vs not having that inventory.
If you're in an area with 100s of hits in a square mile, you have no real leverage here by definition.
I think that would be very hard to crack down on. Hotels are listed on multiple booking sites and have their own site as well. As a property owner I would never want to be stuck with only one rental channel, if one tried to force me I would simply stop using it.
Hotels/hostels are not supposed to not list on AirBnB I thought. (I stayed at a very low-end hotel booked from AirBnB in, and I was very disappointed. When I book an AirBnB I hope for a real host, conversation, see how they live and what I got was a stupid hotel... but I digress.)
They can choose to delist the properties that have a separate channel, by crawling the web, or just querying google, query, etc. to find out if the listing is just a shell. Or they can look at the messages to see if the host suggests book instead on this website. (That of course doesn't prevent direct email communication through which the same outcome transpires. It just catches the most naive showroom cases.)
There are lots of 'condotels' in tourist areas, which are essentially full-service condo buildings that explicitly sell on the idea that the owner can use it for their own holidays, then rent it on AirBNB/VRBO in all the other weeks of the year. These condotels almost always have their own booking site.
I’ve definitely stayed at B&Bs that I booked through other means and I found out by chatting with the owner sometimes used AirBnB as well. No idea what is officially allowed. But if I were renting I’d need a real good reason to grant an exclusive.
VRBO seems like the AirBNB of an older generation. My parents and their friends use it exclusively. To the point they never have visited AirBNB by web or app.
VRBO has been around since 1995. My parents had their vacation condo listed on there for a long time. The reason you don't hear about them is that they weren't controversial with sharing out rooms and didn't invest in pretty design.
I really think they dropped the ball when Airbnb moved into the space
It depends a lot on the area you're visiting. VRBO has a much larger supply in certain areas, especially traditional resort type destinations like ski towns or beach towns.
Maybe it's geographic. I've used VRBO exclusively and never found a comparable option on Airbnb. VRBO fees to renters seem lower and you feel closer to the property owner with less glam or polish from the agent obscuring the realities of the property.
VRBO feels like craigslist, while airbnb feels like eBay to me.
VRBO basically is what people used before Airbnb. As a long time user of it I never really understood the Airbnb hype. I guess Airbnb penetrated markets due to hype and marketing which vrbo was unable to. But they are basically the same thing for end users.
I used OwnersDirect once (part of the same group). I just visited it, HomeAway and Vrbo and they all look like Airbnb clones now. At the time I had to pay for the villa by a bank transfer directly, they didn't handle any of that (and I had to pay the international fees which was rubbish) but I'm guessing they've probably got the same payment experience now too.
So the question is are laws sacred things that should never be broken. I don't think so. I think there are cases that people should break the law so we progress.
In other words, our laws are not the word of God (so to say), they are the word of men (genderless ofc). Good things can happen when you break them and demonstrate how much value can be unlocked or how much a better society we can become without them. By ignoring the laws you ultimately can become a force for change of the law. Those laws usually are there for a reason, of course. But the point is that sometimes those reasons are antiqued or irrational.
I don't know about the case of AirBnB and zoning laws. Don't know enough to have an opinion there.
Well, something that happened where I live (Portland, OR) is that AirBnB became popular enough that people and businesses would buy houses to rent out. One of the consequences was that, because it was so profitable, these renters could buy more, driving up the cost of housing and contributing to an already bad housing shortage. It became very difficult to find an affordable apartment, and it's still pretty tough. I feel like that trade-off (people who own those properties make more money, visitors have a few more places to stay) is skewed pretty heavily in favor of the property owners. And the businesses making that money could easily come from out of state, draining money from the local economy.
You are supposed to have a license (Accessory Short-Term Rental permit) for AirBnB rentals, and it has to be your primary residence. But people still skirt this law.
To summarize your points AirBnB has these effects: 1) more supply & cheaper options for travelers/short-term renters 2) higher rent and less supply for long term renters. 3) More income for property owners (A more varied demand) 4) Less business for hotels/established incumbents.
It's a very hard tradeoff. I can't tell how to argue for one side or the other. Should we use regulation (or enforce existing ones) to help the long-term renters at the expense of travelers? Or should we let free unregulated market roll?
It's not clear-cut case for me either way.
I feel like for WeWork style business, it should be a race to the bottom (in terms of prices). Once the idea is validated, there would be (as there are) lots of local clones that would try to make a more cool indie version of WeWork. The resulting price-cutting and heavy competition make business rather difficult.
As such, I'm not sure where the economy of scale will come from and why should this be a sustainable business? To be fair, Starbucks and other coffee/restaurant chains also have similar general business characteristics.
Along this note, I don't understand why WeWork is a venture-backed business as opposed to a normal bootstrap business (with more emphasis on early cash-flow and less on growth) as it's usually the case with the above type businesses.
There's a certain network effect, at least for some people: you can just take your business on the road and work in a different city every week.
With the name recognition they're getting, they may also get really good deals on office space: I've personally witnessed a co-working space gentrifying a whole city block. The whole block was also owned by the same investor, who apparently gave them a 50% rebate on a ten-year contract. She has easily gotten a 10x ROI on it, since rents in the rest of the buildings tripled within two years.
That's a great point. Hip, cool, modern entrepreneur types can easily work as agents of gentrification which as you pointed out can lead to massive increase in wealth for nearby properties owners (at the expense of previous occupants/patrons, some will argue).
The decreased wealth of occupants is the entire source of the landlord's increased wealth when rent increases, are there really people who will argue that isn't the case?
I guess purely logically, you are right. But I would be against the underlying theme behind that general type of thinking. If you take that line of thought and extrapolate it, it starts getting close to a socialistic/anti-economic progress point of view.
The answer is probably the same as it is for Starbucks et al: execution is a lot more difficult than it looks like from the outside, and nailing scalable execution is an order of magnitude harder yet.
I wonder what will happen to the Facebook the company as this progresses. It would be quite funny if the main product of Facebook becomes Instagram -- and the original product fades.
One thing we might be discounting though is other nations. (I assume you live in N.A.) I don't know how widespread Facebook is elsewhere. It could be that Facebook gives up on N.A. market and let Insta handles their interest there and focus itself on emerging markets.
A probable role Facebook may take in future is providing digital "passport" kind of identity. The Chinese Facebook clone is doing it already and the Russian Facebook clone does too though in much smaller degree.
I live in Europe, not N.A. We use messengers a lot here and many people still use Facebook. I've heard that people use SMS and MMS much more actively in the USA (I have never used MMS and have stopped using SMS years ago).
That's what we call "texting", but very often texting isn't done using SMS. With the iOS market share hovering slightly below 50% in the US, it's iMessage that is the main messenger, at least where I live. The iOS Messages app seamlessly switches over to SMS when the receiver doesn't have iMessage - and shows a visual distinction - but in my case, that's a minority of the people I text with.
I don't use Whatsapp nor was I ever invited by someone to use it, but I heard it's basically the iMessage equivalent in the EU and in other places where the iOS market share is lower.
I don't think so. A/B testing is like gradient descent which is a greedy algorithm. You move in the direction that locally looks best. Evolution on the other hand allows for suboptimal species to persist for enough time to let them develop their advantage. (In the language of optimization evolution allows you to go past the local optimal and reach global optima by allowing you to move in non-optimal direction -- as long as the move is not catastrophic.)
Nope. A/B testing depends on what you choose to mutate. The problem is that we humans intentionally change the things. Nature randomly changes the things.
You’re not going to get to a global optimum driven by human choice of what to test (only local optima at best) unless the human setting up the tests is some sort of sage.
Do you Nature will eventually hit on a global optimum? If so, that belief is opposed by what biologists know about developmental and evolutionary pathways.
For a trivial and well-known example, there is now no developmental or evolutionary pathway that will lead to a vertebrate eye where the nerves run behind the retina. As a result every last vertebrate has a blind spot where all of the nerves dive through the retina. This makes our eyes less efficient because the nerves block out light.
We're therefore stuck with the first design of the vertebrate eye and can't change it. There is no pathway to the more sensible design of the mollusk eye.
This is but one of many examples. For another one, relative developmental timing of growth is fixed across vertebrates. For example the "hand" grows before the "arm". The result is that pterodactyls, whose wings are entirely hands, could fly from birth. But birds (whose wings are arms) and bats (a mix of the two) can't fly from birth. No matter how desirable an evolutionary trait that may be.
> Do you think Nature will eventually hit on a global optimum?
No. At least not in the way described. Optimal as considered by whom? Us? We look at something and say this would function better than that, but is something that is optimal perfect? Basic microeconomics shows that perfection is sub-optimal due to the law of diminishing marginal return. Perfection is wasteful and unsustainable. What do we know to be optimal? From what I know, “adequate enough to reproduce” may not be perfect, but it might be brushing up against optimal in its “good enough and no more” / lagom nature. Mollusks have mollusk eyes. Can we know that humans would be better off with mollusk eyes without rewriting our evolutionary lineage for us to be more like mollusks in other imperfect ways?
In my view philosophy is important in and of itself, because these are natural questions about our existence which many brilliant people have thought about and written on for centuries. And these are very worth reading.
As an example of this general litmus test for a theory see e.g. Eddington's confirmation of GR: https://en.wikipedia.org/wiki/Tests_of_general_relativity#De... . If there are hitherto unknown phenomena in DL predicted by this theory then I'd stand corrected and concede that there may be something to these theories.