In terms of melodies I think trainning off lead-sheets is far better than raw midi. The reason being is when humans generate melodies they can anticipate implied harmonic structure. i.e.Chords. You folks might be interested in a leadsheet dataset i'm creating for the purpose of generating melodies: https://github.com/shiehn/chord-melody-dataset
Ah ok, I really just made an assumption that the goal is to generate convincing harmonic music. In this case after spending a couple years fighting with that problem I've been reaching a conclusion that the chords and melody are absolutely by far the most difficult & 'human' aspect where as counter-point/polyphony are actually very mechanical & really just support the chord/melody structure. Recently I've been having very good results by first generating chords/melody with RNN's then having subsequent processes generate supporting harmonies.
These are results from starting with just a chord progression & stacking ontop of it:
http://signalsandsorcery.com About 5 months ago I decided to pivot & start with a melody & chords. I have not yet stood up the system that starts with chords & melody. So the answer is not yet.
Ya, good question. I doubt there is a single method. I suspect pre 20th century single note melodies were often sung and then following the strict rules of counterpoint harmonies were added. Its seems like pop music is often a chord progression first and then a melody added second.
Next step will be to clone ranking from some well-known music critics which would actually make it work this time. Behavioral cloning is now used heavily in computer vision, haven't seen it used in music generation yet, but I expect it to be groundbreaking as algorithm would be able to discern certain qualities that pure generating sequences won't.
