Its not just you. Mango is part of the Cashew family which also includes Poison Oak(!) The skin of the fruit and especially the area around the stem often contain these same toxins. Ive had the same reaction to Mangos before with swollen lips and a red itchy rash. However, I can drink Mango juice and eat canned Mango with no ill effects. If you like Mango enough that you want to try it again, peel it first(I recommend wearing rubber gloves!!!!!) then cut it into pieces before you eat it.