There are some people doing that, it takes a ton of work and the device will probably be dead by the time you are finished. Some links in the articles section here:

https://wiki.debian.org/Mobile