I guess if you hurt it and put too much strain on the penis. The way i see it is it's a similar concept just like all your other muscles you train in the gym, you tear the muscles and it rebuilds thus becoming bigger and stronger than previously.
Penis on the other hand you could tear something inside which could/would give you a small looking size due to it but once it heals it should be back again ready to knock another girls world and hopefully end up gaining a small bit of length or even girth with it.