It can allegedly cause some brain developmental issues before if used before the age estimated that the frontal cortex is developed (25ish) frontal cortex is responsible for impulse, and cognitive behaviour. On top of that I know short term memory loss is an effect, but it's reversed if stopped. Permanent damage seems to be difficult to find. Smoking the flower can cause some respiratory irritations via burning, but all studies I have seen that followed smokers showed little to no damage to health with heavy smoking long term.
It's safer than sugar, alcohol, and cigs which a high percentage of people aren't concerned about. It does more good things than bad as well.