> Having two IRs (or even having a 1st-stage IR for each frontend language) sounds like something that a compiler project would want to avoid in the long term
Can you please share the reason behind why it should be this way ?
I'm not a compiler engineer, so I'm just sharing my own point of view, but to me it sounds like a heavy maintenance burden. Whenever possible I would want to avoid having multiple versions of an important subsystem in any large program that I worked on.
This is true, I tried to make gccrs only have the AST and go from that stright to GCC GENERIC. This had a lot of problems for Rust in my opinion.
Many things in Rust are syntactic sugar and can be handled by desugaring the AST into another IR so you dont even have to think about it in other passes. The main issue for me is how complicate the type inference is.
So if i wanted to use GCC GENERIC for type resolution for this example:
```
let a;
a = 123;
let b:u32 = 1;
a += b;
```
How do you resolve the type of 'a' you must use inference variables and then update the TREE_TYPE as you go so this means walking the tree's over and over again as type information is gathered over time on the inference variable. Using a separate IR and using id's and side tables makes all of this much much more simple for Rust.
Firstly, Congratulations. This is no trivial accomplishment. Here are the questions I have.
1. Normally, cars come with their own ECUs for the engine. Interfacing with them requires knowledge about the underlying firmware used afaik. Did you happen to reverse engineer the firmware so that you could interface your own electronics with the provided ECU ?
2. How did you make the car believe that your 6 speed automatic transmission was a 4 speed automatic ?
3. > "the ability to go ahead and change the way it works whenever I want." - Could you please elaborate what you mean by this ?
Again, I am amazed by what you have accomplished. My inner mechanical engineer is so keen to know how this all came together. I think everyone would definitely appreciate if you could provide a rough overview of how you achieved these things. Thank you :)
1. I don't interface to my car's electronics over serial or CAN bus. Instead, I interface by emulating actuators and sensors the 4 speed gearbox computer expects.
2. I'd like to introduce you to "manualisation" [0]. It's where the solenoids of the 4 speed are wired directly to the gear selector switch. The 4 speed computer is surprisingly tolerant of this, and it forms the basis of my interface.
My interface provides enough information to the 4 speed computer to stop it from erroring out or putting my car into limp mode. However, it is aware that something has changed because changes in engine RPM do not correlate with its shift commands.
3. I mean in terms of software. I can modify the behaviour of my box more than a standard tune can. I can add a 0-100KM/h tracker to my system, or shift lights, I can fundamentally change the decision making process in shifting (for the moment it's torque based), etc.
I could go out on a limb and a GPS module and a map; it'd select a gear based on the corner angle and speed or so.
Thank you for kind words :) If you have any other questions, ask away!
Thank you for your reply. That is quite enlightening. I am quite surprised that the Transmission Control Unit(TCU) is tolerant of the solenoid modification. However, I have two questions though.
1. So based on my understanding, engine speed sensor readings are sent to the TCU which in turn regulates the solenoids(There might be other sensors as well but for my question, I think they aren't that important). So in order for your modification to work, you have to get these CAN frames, understand the reading, regulate your 6 speed auto gearbox and then create a CAN frame that would imitate the sensors of the 4 speed auto gearbox(to basically tell the TCU that the gearbox has been regulated correctly). You can in principle neglect the actuator commands that are sent by the TCU to the solenoids since you are regulating it on your own. However, in order to do this, you have to understand the CAN frames being used and also the protocol used(address based or message based). How did you figure this out ? Or is my understanding wrong here ?
2. Did you use FreeRTOS for your STM32F407 component ?
I really appreciate you taking the time to answer the questions :)
I was quite surprised, too. Maybe it was a little gift from Holden to the burnout bogans?
1. My apologies, I've confused you. CAN communications are purely internal for my system, it doesn't communicate via CAN to the car's computers.
The sensors and actuators give or accept voltage signals, and my controller inputs or outputs those accordingly. For example, the 4 speed has some pressure sensors to monitor the valve body behaviour, and I send some HI or LO voltage signals in emulation of it.
2. I do use FreeRTOS for the 407. I don't need to, since I use a super loop and all my memory is either static or allocated on the stack.
Ah okay. This simplifies the setup a lot. I had done some testing and validation of ECUs 9 years ago. So my questions were based on the understanding I had developed back then. Thank you for clarifying it. Again, awesome work :)
If my car relied heavily on CAN (like the next generation Commodore) I probably would have sold it.
I love CAN bus (for my own designs), but I don't see the effort of attempting to reverse engineer GM's format as worth it, especially since I had no prior experience with CAN.
Thank you :) I've got another D automotive project which I hope to write about soon.
I was wondering whether it is be possible for you to provide an overview of different methods that you think might have a better shot at replacing backpropagation algorithm?
Sure. First of all, I want to say that backprop, by which I mean reverse-mode differentiation for computing gradients, combined with gradient descent for updating parameters, is pretty great. In a sense it's the last thing we should be trying to replace, since pretty much the whole deep learning revolution was about replacing hand-designed functions with ones that can be optimized in this way.
Reverse-mode differentiation has about the same time cost as whatever function you're optimizing, no matter how many parameters you need gradients for. This which is about as good as one could hope for, and is what lets it scale to billions of parameters.
The main downside of reverse-mode differentiation (and one of the reasons it's biologically implausible) is that it requires storing all the intermediate numbers that were computed when evaluating the function on the forward pass. So its memory cost grows with the complexity of the function being optimized.
So the main practical problem with reverse-mode differentiation + gradient descent is the memory requirement, and much of the research presented in the workshop is about ways to get around this. A few of the major approaches are:
1) Only storing a subset of the forward activations, to get noisier gradients at less memory cost. This is what the "Randomized Automatic Differentiation" paper does. You can also save memory and get exact gradients if you re-construct the activations as you need them (called checkpointing), but this is slower.
2) Only training one layer at a time. This is what the "Layer-wise Learning" papers are doing. I suppose you could also say that this is what the "feedback alignment" papers are doing.
3) If the function being optimized is a fixed-point computation (such as an optimization), you can compute its gradient without needing to store any activations by using the implicit function theorem. This is what my talk was about.
4) Some other forms of sensitivity analysis (not exactly the same as computing gradients) can be done by just letting a dynamical system run for a little while. Barak Pearlmutter has some work on how he thinks this is what happens in slow-wave sleep to make our brains less prone to seizures when we're awake.
I'm missing a lot of relevant work, and again I don't even know all the work that was presented at this one workshop. But I hope this helps.
Yes, "tuning for criticality" was the paper I was thinking of ! But I'm afraid I'm a dilettante when it comes to neuroscience. I basically just know the basic theories about consolidating learning during sleep.
Thank you for your answer. It appears to me that we are trying to achieve an algorithm that has better time complexity than the one that we have right now(reverse mode differentiation with gradient descent).
Is it possible to combine these methods in a straight forward manner with methods that try to reduce the space complexity? For example, Lottery ticket hypothesis(https://arxiv.org/abs/1803.03635) seems to reduce spacial complexity(Please do correct me if I am wrong).
Also, based on my rather poor and limited knowledge, it appears to me that set of proposed methods that reduced space complexity and set of proposed methods that reduce time complexity are disjoint. Is that the case ?
Thanks for your question! But as I said, no one is really worried about the asymptotic time complexity of reverse mode differentiation, although there is scope for improving constants). The main scope for improvement is in the space complexity.
There is a lot of work on trying to speed up optimization, for example the K-FAC optimizer by Roger Grosse that uses second-order gradient information in a scalable way.
The lottery ticket pruning strategies do reduce space complexity, but I think the main reason people are interested in it is to reduce training time complexity, or deployment memory requirements, but not so much training memory requirements.
As for whether memory-saving and time-saving approaches are disjoint, many methods (like checkpointing) introduce a tradeoff between time and space complexity, so no.
(Lottery Ticket, to date, produces small networks ex post facto... You still have to train the giant network. There's also some indication that it's chancy on 'large' datasets+problems. https://arxiv.org/abs/1902.09574 )
This sounds rather strange and sad. I had assumed that their account creation method was motivated by avoidance of creation of spam accounts. If there are other more reasons they should probably mention it in their website. I hope someone from lobsters' moderation team sees this and I hope its not some "exclusive club" since I like some aspects of their website like tags, "what are you working on this weekend" thread etc.,
>...I had assumed that their account creation method was motivated by avoidance of creation of spam accounts..
Me too. But it seems that, at least some of their members [viz. the person I chatted with on IRC] are using their ability to hand out invitations as a bit of a power trip and to mess people about.
To expand on what I wrote before [I was on my phone then, so keeping the typing as brief as I could]. My IRC conversation with the Lobste.rs person, who I shall henceforth refer to as 'XXX' [not for any reasons of anonymity --I've just forgotten their IRC handle!] took the form of:
* Some discussion about the lobste.rs site [presumably to show I actually knew what it was about] and had frequented it.
* Some discussion as to whether I had any examples of 'stuff' I'd written to show them. I posted a link to one of my blogs and XXX disappeared for a few minutes, then returned and said something along the lines of "You have some interesting stuff there".
* XXX then asked how I could prove that I was the author of that blog. After a bit of back and forth suggesting different possibilities, we agreed I'd put a message on the blog in some way.
* So, I went off and updated my blog, adding a "Hello lobste.rs!" message in an X-Header
* I then asked XXX to check the blog again and look for the message in the X-Headers, which s/he did.
* XXX then [as mentioned before] then returned, thanked me for the conversation, added "..but I'm still not giving you an invitation!", said goodbye and left the channel.
I felt a bit humiliated. Almost like I'd sat through a job interview and then been rejected without explanation. I also felt annoyed with myself for having jumped through hoops to try and persuade some anonymous person on the internet that I was good enough to join their club.
On the face of it, the concept behind Lobste.rs seems to address some of the shortcomings of the "benevolent dictatorship" of HN by giving admission and moderation power 'back to the people'. But, in practice, this also gives significant powers to every member of the site, irrespective of their being mature enough to be worthy of this trust.
Wow! This sounds like a nightmare and honestly very disrespectful. I have nothing to show for since I keep my presence on the internet to a minimal. I guess i will be denied an account in a manner worse than you(This may sound like a knee-jerk reaction. However, it is one of the many possibilities and I am rather willing to be pessimistic than optimistic).
On the personal side, here is an hug(or a professional equivalent if you prefer that) to you stranger. I hope you have a great day.
I am wondering about the reasons for these issues that you mentioned. Could you please mention them here ? It might serve as a nice pointer to do better hardware design debugging I believe(I have close to zero experience in this. Hence the interest).
Also, are there any formal systems that verify your TTL-level designs ?
Sure. There are many reasons why a real circuit falls short of the idealized abstraction.
A truly squarewave clock, or any rising/falling edge that happens in zero time would have infinite bandwidth, and no real circuit can have infinite bandwidth because of capacitance, inductance, and resistance. Even though the power rails of a TTL chip are 0V and 5V, when a fast transition happens, the signal can experience reflections because the impedance of the driver and the copper trace are different, and because the end of the copper trace isn't terminated with impedance which matches the trace. As a result, the edge takes finite time to transition from 0V to 5V, but the signal can actually exceed 5V because of a reflection. Likewise, falling signals can go below 0V.
Also, the signal at one end of a wire will not be the same as the signal at the other end of the wire. It isn't even just a simple time delayed version. Eventually they'll come to equilibrium if the signal is stable long enough.
OK, another defect. Imagine a TTL part with 8 buffers in it, say a 74ACT244. Say all the outputs are low, and at some moment seven of those inputs switch from low to high. The corresponding outputs must drive enough current to switch their loads from low to high as well. Each of those outputs draw current to charge their load from 0V to 5V, and all of the current flows through a single power pin and a single ground pin. Because of of non-zero resistance and inductance, the voltage seen by the transistors on the chip is shifted slightly from the voltage seen on the PCB. That can cause the output which isn't changing to sink/source current (depending on the direction of the change) causing its output to bounce a bit. Imagine having four people on a trampoline, with one standing still and three jumping at the same time. That "standing still" person will still bounce up and down because of the others causing their common ground reference to shift.
Another defect. A chip's spec sheet lists a bunch of requirements for proper operation and guarantees of the chip's behavior if the requirements are met. Say a flip flip says it has a 5ns setup time, that is, the data in to the flop must be held stable for 5ns before the rising edge of the clock arrives. If this requirement isn't met, the chip's behavior is unspecified (it might capture it, it might not, it might capture it but it might take an arbitrarily long time to appear on the output, or the output could even oscillate). During the design phase one adds up the worst case propagation times for all paths from point A to point B and makes sure there is at least 5ns to spare to guarantee the requirements. But in real life, a clock trace runs all over the board and has many loads, each presenting some amount of capacitance. Kind of like relativity, different chips will see the rising edge at different times. Looking at a given chip, the clock edge and data inputs are each transitioning from high to low or low to high over the course of a nanosecond (and often more). At what point on that slope is the signal really a "0" or a "1"? Sometimes those signals are clean transitions because of reflections, it might go from 0.7V before the clock edge, to -0.5V after the clock edge, to 2.2V (neither logical 0 nor 1), stay like that for a few ns, then move to 4.2V for a few ns, then 4.0V for a few ns, then 4.6V.
The most maddening is the fact you can't always trust what the oscilloscope tells you. The oscilloscope has finite bandwidth; 100MHz scopes were common in the 80s and early 90s when I was using them. The 200MHz scopes were a lot more expensive. You might think, hey my clock is only 25 MHz, 100 MHz is plenty ... but a fast signal edge contains harmonics that are a multiple of that 25 MHz base frequency, so the scope tends to act as a lowpass filter. The scope probe is typically more than a meter long, and there is a clip on the probe that must be attached to your circuit ground. That is a 2m loop of wire, and in the best case it would take a signal a few ns to travel the length of that wire, and any stray EM fields can induce signals on that probe that aren't in the circuit (though obviously the probe designers take pains to minimize such effects). When debugging and a signal looked unexpectedly weird, a common tactic to avoid sinking too much time chasing phantoms was to change the probe, change the channel, or simply waggle the lead wire around to test the sensitivity of the displayed waveform on stray capacitance and such.
Testbenches always had a few cans of refrigerant on hand. If you suspected a chip might be flaky, probe its output then spray it and see if the output is affected by the chip temperature.
Your explanation is quite amazing. In my undergrad, we had learned about Transistors, TTL, flip-flops etc., but as individual building blocks. Your reply paints a very good picture about what can go wrong they play together. Is there a book that you would recommend to learn more about this ?
Also, Honestly, I thought reflections would be negligible when we are dealing with mA and 0-5V ranges. But your reply makes me think that there is more to it
Finally, thank you for your reply. I gonna come back to this answer in the future when I have a lab setup that is more than a solder and some screwdrivers :)
Oh, I forgot to mention that when you probe a circuit, the presence of the probe can affect the operation of the circuit.
On RF designs it was not uncommon to find seemingly needless small valued components at certain probe points. The idea was these modeled the reactive load of a probe. When you needed to probe it you'd remove the components and attach your probe and the circuit see approximately the same load so you could have more faith that what you were seeing more closely matched what was happening when you weren't looking.
I have seen such small valued components, looked their values up and thought, "this is insane".
(I wish I could say I thought, "I must be dumb, someone must be much smarter than I am", but if I hadn't embodied a certain kind of hubris I would not work in software.)
> The frustrating thing about my project is the sheer amount of computation required. I really don't need a direct photogrammetry capture of a trail, an approximation would be fine to some degree. But I take like ten gigabytes of video data and then process each frame to find keypoints, run correlation on all these points, and all this (using COLMAP). This stuff can take days to process on my desktop.
If the hiking trails are accessible enough, you should have a look at SLAM technique. SLAM allows you to create smooth and rough approximate map of the environment through which you navigate your camera. Colorization of this map could be done by a GAN(might be an interesting side project).
Thank you very much! It has been a long time since I've looked at visual slam actually. That Omnidirectional LSD-SLAM looks really nice. Their code repo has been untouched for six years, but this still makes me realize I need to use vslam! I just found this recent work which seems really useful.
I feel like vslam could be the the first step in a post-processing pipeline that would reduce a lot of the computational complexity of solving large maps. Once I can easily make large maps I can build simulated environments and use those for training an agent.
