There are two types of synapses:
- Chemical synapse
- Electrical synapse
The first one is the one you are asking about. The second one corresponds to the faster synapse you are imagining. It consists of two neurones connected by a gap junction. Gap junction form a cytoplasmic bridge between the neurones and thereby allow electrical signal to directly go from one neuron to the next through ion diffusion without converting to chemical signal.
The second one implies release of chemicals (neurotransmitter) in the synaptic cleft, the extracellular space between the two neurons. It is indeed slower than the electrical synapse. However, unlike the electrical synapse, it enables a variety of postsynaptic effects. Some neurotransmitter elicit excitatory effects, some elicit inhibitory effects (electrical synapses where the presynaptic side is an axon only allow excitatory effects). Some postsynaptic receptors to neurotransmitter are ionotropic whereas some are metabotropic. The former are neurotransmitter-gated ion channels, which means that they let specific ion cross the postsynaptic plasma membrane when they bind neurotransmitter. The latter trigger intracellular chemical reactions, such as production of cAMP, which is a companent of a signalling pathway and can for example influence the expression level of specific genes.
This more diverse set of postsynaptic response enabled by chemical synapses explains why they dominate our nervous system.
Reference: Purves et al., Neuroscience, 5th ed., 2012 ISBN 978-0-87893-695-3