Touches of evil genius - how the best horror movies delve deep into our primordial anxieties

Some of the greatest and cleverest movies made have been horrors, and the best of them have more to tell us about people's insecurities than the monsters they concoct. Ghostly and supernatural folk stories, always a great favourite in this country, were traditionally an attempt to control primordial anxieties, and when motion pictures arrived in the 1890s, film-makers were immediately drawn to them.

In the 1910s, pioneering French director Georges Méliès moved beyond the floating heads and dancing skeletons he'd playfully rendered in his early films to more sophisticated tales involving Faustian pacts with a demon often played by Méliès himself. The best of them was probably Le Chevalier des Neiges, in which a princess is kidnapped by demons and must patiently await rescue.

The visual potential of supernatural tales was obvious, but was never fully realised until the 1920s, by the German expressionists. Movies like Nosferatu, The Golem and The Cabinet of Dr Caligari would coin much of the visual language used by later film-makers, and are among the greatest horrors ever made: in fact American film critic Roger Ebert called Robert Wiene's Cabinet of Dr Caligari the "first true horror film".

Its visual style was certainly unforgettable, with weirdly distorted theatrical backdrops adding to a nightmarish, otherworldly feel. The town of Holstenwall is plagued by unsolved murders, and a young man called Francis becomes obsessed with finding out who's doing them. He meets the fairground performer Dr Caligari, who works with a somnambulist called Cesare who can tell the future and predict the exact moment of your death. Turns out there's a reason for that.

There was substance in Dr Caligari as well as style, and the Weimar horrors dipped backwards into folk tales and central European ghost stories that modernity thought it had left behind. FW Murnau's 1922 film Nosferatu was strangled at birth when Bram Stoker's estate decided that it strayed too close to the plot of his novel Dracula. Only a few prints survived, and over time its quality has been appreciated.

The vampire in Nosferatu has little in common with the suave bloodsucker played by Bela Lugosi and Christopher Lee in later films: this guy is a hideous, goblin-like Transylvanian count. In one unforgettable scene we watch his twisted shadow slowly climb a staircase leading to a woman's bedroom: this wasn't throwaway horror, it was bloody terrifying.

Through the 1920s, the dreamy Gothic Weimar style spread to Hollywood, where the term 'horror movie' would first be coined. They fainted in the aisles when Lon Chaney's face was revealed in The Phantom of the Opera (1925), his features distorted by wires and pins into a skeletal rictus. Still more elegant chills would be provided by the classic Universal horrors of the early 1930s, with James Whale's Frankenstein and Bride of Frankenstein brilliantly providing an archly modernist take on Mary Shelley's 1818 novel.

Horror was developing its own vernacular, and its best exponents didn't need big budgets. An exercise in minimalism and the power of suggestion, Jacques Tourneur's 1942 film Cat People was made for less than $150,000 using discarded sets from other movies, but would have a lasting influence. Simone Simon played a Serbian woman called Irena Dubrovna who has obsession with all things feline. When she marries a rich engineer, Irena becomes insanely jealous of his pretty secretary, and a vicious cat-like identity is revealed.

While Hollywood horror would be made ­unrespectable by the atomic-inspired excesses of the 1950s B-pictures, in Europe they remained a valid means of expression for serious filmmakers. They don't come much more serious than Henri-Georges Clouzot, the expressionist visionary who in 1955 made one of the most unsettling, intelligent horrors of them all.

In Les Diaboliques, the director's wife, Vera Clouzot, played Christina, the abused spouse of a loathsome school headmaster called Michel who's also having an affair with one of his teachers, Nicole (Simone Signoret). The two women get together and decide to rid themselves of their tormentor, drowning him in a bathtub and dumping the body in the school's swimming pool. But when the body disappears and the women hear reports that Michel has been sighted alive, they panic.

Clouzot's grim masterpiece is said to have inspired Alfred Hitchcock to make Psycho, a project considered so sordid by his studio, Paramount, that they refused to finance it. Hitch used his own money, and shot in black and white to keep his budget down and make his shower scene look less grotesque.

Janet Leigh played a young woman on the run who pulls into the wrong motel, and Anthony Perkins was scarily convincing as Norman Bates, the mother-obsessed psychopath loosely based on real-life serial killer Ed Gein. The film that launched the slasher craze is so much better than anything it inspired.

British horror came into its own in the 1950s and 60s, and while Hammer Studios were pumping out cheap and cheerful chillers, others were aiming higher. Michael Powell's illustrious career was effectively ended by the vicious reception his 1960 film Peeping Tom provoked. An innovative and fearless horror, it told the chilling story of Mark (Carl Boehm), a technician at a movie studio who kills women with a spike concealed in his camera. Powell intended the film as a withering critique of voyeurism in cinema: no thank you said the critics.

They were kinder to The Innocents, Jack Clayton's elegant, brilliant adaptation of Henry James' ghost story Turn of the Screw starring Deborah Kerr as Miss Giddens, a prim governess who notices that the orphaned niece and nephew she's minding seem to be communing with invisible forces.

In America, the genre was given a shot in the arm by two films that dealt with demonic possession. Roman Polanski's Rosemary's Baby (1968) starred Mia Farrow as Rosemary Woodhouse, a naïve young bride who's delighted when she moves into a plush apartment in uptown New York. But the neighbours are shifty, and when Rosemary gets pregnant, she realises she's carrying the spawn of Satan.

More unsettling still was William Friedkin's The Exorcist (1973) which is still one of the most frightening films ever made. Ellen Burstyn is a busy actress who calls in the priests when her teenage daughter begins having spasms and speaking in tongues. It's chilling stuff.

In the 1980s and 90s, and thanks in part to Hitchcock, a slasher trend drained Hollywood horror of its seriousness and originality, and things got worse in the early 2000s when the so-called 'torture porn' aesthetic of films like Saw and Hostel replaced it.

Since then, however, and possibly as a reaction to all that nastiness, horror has moved quickly on into what could tentatively be described as a new golden age.

In recent years the wit and originality of Jordan Peele's Get Out and Us, the boldness of Ari Aster's Hereditary and Midsommar, the sheer creepiness of Jennifer Kent's The Babadook and the sly humour of Ben Wheatley's Kill List have reminded us of what the horror genre can achieve if it abandons cheap tricks and gratuitous gore and aims a little higher.

Top five horror fims for Halloween

Nosferatu (1922)

Though he died young and only made eight films, FW Murnau will always be remembered for Nosferatu, the most boldly original, visually beautiful and unsettling vampire movie of them all.

Don't Look Now (1973)

Nic Roeg scared the Jesus out of early 1970s cinemagoers with this dark tale of a couple who move to Venice to recover from the tragic death of their child, who they start catching glimpses of down side streets. Terrifying.

The Exorcist (1973)

You never actually see much in William Friedkin's The Exorcist, and indeed for long periods nothing much happens at all. It's the tension, and that ominous music which plays on your imagination and makes you start seeing things.

The Shining (1980)

Stephen King famously loathed it, but Stanley Kubrick's splendidly gloomy epic may just be the greatest horror of all. When Jack Nicholson's would-be writer becomes caretaker at a snowbound hotel, something snaps. Bad news for his wife and kid.

Get Out (2017)

Jordan Peele's sly and witty horror brilliantly illustrated how racism can lurk behind even the most well-intentioned white smile. Daniel Kaluuya's streetwise photographer smells a rat when he visits the plush country home of his white girlfriend's parents.