Summary of compounds. The chemical structure, molecular weight (MW), and α4β2 nAChR EC₅₀ are presented for all test compounds: a nicotine, b cotinine, c anatabine, d methylanatabine, e anabasine, f nornicotine, g metanicotine, h acetylcholine, and i AZD1446. The EC₅₀ values are presented in mean ± SEM. ^aPotential partial agonists are cotinine (59%), methylanatabine (26%), anabasine (7%), nornicotine (44%) (Online

Source 3). The percentages in the parentheses are calculated based on the maximum receptor activation by nicotine as 100%

α4β2 nAChR activation by alkaloids in vitro. Dose–response curves for α4β2 nAChR activation by respective alkaloids in vitro are presented for a nicotine, b cotinine, c anatabine, d methylanatabine, e anabasine, f nornicotine, g metanicotine, and h AZD1446 in black lines. The grey lines represent the fits to the Hill equation for acetylcholine. The average values were fit to the Hill equation. The EC₅₀ values are presented in Fig. 1. The averag normalized current response is plotted as a function of the maximal current to acetylcholine (I_maxACh) (n = 3–11). The acetylcholine curve was replotted for each graph. Data are shown as mean ± SD

Time spent at the bottom of the tank during NTT. Time spent at the bottom of the tank during NTT is presented for a nicotine (0.3, 1, and 3 mg/L), b cotinine (30, 100, and 300 mg/L), c anatabine (0.3, 1, 3, and 10 mg/L), d methylanatabine (1, 3, and 10 mg/L), e anabasine (0.3, 1, and 3 mg/L), f nornicotine (3, 10, and 30 mg/L), g metanicotine (30, 100, and 300 mg/L), h acetylcholine (30, 100, and 300 mg/L), and i AZD1446 (30, 100, and 300 mg/L). Black dashed lines = control; light blue = lowest concentration; blue = middle concentration; purple = highest concentration. For anatabine only, red is the highest concentration. *p < 0.05; **p < 0.01; ***p < 0.001 compared to the vehicle control. Data are expressed as mean ± SEM

Effects of compounds on total distance traveled. Total distance traveled in the entire tank during the NTT is presented for a nicotine (0.3, 1, and 3 mg/L), b cotinine (30, 100, and 300 mg/L), c anatabine (0.3, 1, 3, and 10 mg/L), d methylanatabine (1, 3, and 10 mg/L), e anabasine (0.3, 1, and 3 mg/L), f nornicotine (3, 10, and 30 mg/L), g metanicotine (30, 100, and 300 mg/L), h acetylcholine (30, 100, and 300 mg/L), and i AZD1446 (30, 100, and 300 mg/L). Black dashed lines = control; light blue = lowest concentration; blue = middle concentration; purple = highest concentration. For anatabine only, red is the highest concentration. *p < 0.05; **p < 0.01. Data are expressed as mean ± SEM

α4β2 nAChR binding by alkaloids in vitro. Dose–response curves for α4β2 nAChR binding by respective alkaloids in vitro are presented for a nicotine, b cotinine, c anatabine, and d methylanatabine. The average EC₅₀ values are included in each figure and summarized in panel (e). n = 3. Data are shown as mean ± SD

Docking positions of nicotine and cotinine at the α4β2 nAChR agonist-binding site. Docking positions of a nicotine (carbon atoms in blue) and c cotinine (carbon atoms in red) at the α4β2 nAChR agonist-binding site are shown. The nicotine pose docked in a similar position as the crystalized pose (carbon atoms in gold), with a binding energy of − 6.3 kcal/mol. The 2D interaction schemes show that b nicotine interacts with Trp156 and Cys199 and d cotinine with Trp156 and Tyr204. The interactions are shown in dashed lines between nicotine or cotinine and the receptor residues. Pink circles represent polar residues. Green circles represent hydrophobic residues

Docking positions of anatabine and methylanatabine at the α4β2 nAChR agonist-binding site. Docking positions of a anatabine (carbon atoms in green) and c methylanatabine (carbon atoms in pink) at the α4β2 nAChR agonist-binding site are shown. The 2D interaction schemes show that b anatabine interacts with Trp156 and Tyr204 and d methylanatabine with Trp156, Tyr204, and Thr157. The interactions are shown in dashed lines between nicotine or cotinine and the receptor residues. Pink circles represent polar residues. Green circles represent hydrophobic residues.