If youre curious about evolutionary computation/algorithms, the Coding Train has very accessible videos and coding tutorials explaining them. There's tons of research combining evolution and learning. Evolution is slow both biologically and in computers, learning is very fast. Combining them is nothing special because /we/ need to define the goals the programs optimize for.

What you're concerned with is called "Open ended evolution" and "open ended learning" describes the more biological concept of agents that create their own goals from their senses - but so far, it seems to be way farther from where we are now with AI.

https://www.youtube.com/playlist?list=PLRqwX-V7Uu6bw4n02JP28QDuUdNi3EXxJ