Molecular determinants of selective ligand binding at the human melatonin receptor

Introduction. We have recently identified a series of substituted isoquinolinones (1) that act as highly selective agonists for the melatonin MT₂ receptor. These ligands represent valuable tools for delineating potential structural determinants that underlie the molecular properties of the MT₁ and MT₂ receptors.

Aims. To identify the key residues located at TM5, TM6 and TM7 of the melatonin receptors that contribute to the isoquinolinone binding pocket and govern subtype selectivity.

Methods. Conserved residues on both MT₁ and MT₂ receptors, including those which had been reported to have essential interactions with melatonin, were subjected to Alanine substitution. Selection of residues was guided by molecular modelling of the MT_1/2 receptors. Wild-type and mutant receptors were examined for Ca²⁺ mobilization upon isoquinolinone stimulation in transfected cells. Expression of receptor mutants was confirmed by radioligand binding assays.

Mutation of an essential MT₂ residue, His208, severely impaired the potency of melatonin but did not affect the isoquinolinones. Mutation of two conserved tyrosine residues on TM7 to alanine (MT₁ Tyr281/Tyr285 and MT₂ Tyr294/Tyr298, respectively) had no effect on melatonin. In contrast, the MT₂-selective isoquinolinones lost their selectivity and became able to activate the MT₁ mutant receptor.

Discussion. Our mutagenesis studies suggest that the isoquinolinones and melatonin utilize different subsets of residues for receptor activation. A proposed binding cavity formed by a large group of hydrophobic amino acid side chains located at TM5, TM6 and TM7, may accommodate the aromatic substituent of isoquinolinone compound. This cavity includes the two conserved tyrosine residues on TM7 which appear to impose steric hindrance at the MT₁ receptor and result in a predominantly MT₂ selectivity profile for the isoquinolinones.