A brushed motor works on a simple 2 phase (negative and positive power supply) DC power system and a brushless motor works with a 3 phase (A, B & C cables) system that also requires the use of a ESC (Electronic Speed Controller).
A brushed motor will require more maintenance, will wear out its brushes through use over time and will produce less power than brushless motors. Although brushed motors were once all that was used for every RC from the beginner hobbyist to the podium of world champion drivers' cars, today they are mainly used in entry level RC products to help keep costs down. These motors can't usually be serviced and so once they die, a replacement is the only way to get your RC running again. With that said, there is still a place for brushed motors in the hobby and that is mainly in RC crawlers. In this genre, there are still quite a few high spec motors available that can be serviced and rebuilt if required.
In the video below Ryan takes us through the best way to clean and maintain a Brushed Motor
Brushless motors are a very different product and have allowed the hobby to expand and grow in areas that would never have been possible with brushed motors. Genres like quads/drones, EDF jets and large scale electric RC were never possible before brushless motor tech advanced to where it is today. These motors produce a lot more power due to their efficiency, require very little maintenance and come in various sizes and specifications that have made them the perfect choice for pretty much every type of modern RC product depending on the scale, speed, voltage and budget required.