It isnt just the penis your resting. When you hit the gym and blitz the muscles than rest afterwards it isnt just the muscles that are resting, but your entire biological systems that are working overtime to manage the massive energy surge placed on it.
The human body is GREAT at keeping its reserved energy, it does not want to part with it and is why you need to literally force growth at the gym and lift that heavy iron, digging deep into your hidden reserves of energy.
Your organs are also resting such as the pancreas, liver, kidneys, heart etc as lots of stress are placed on them when you do gym work. Now as you say Penis Enlargement is different to gym work and one obvious difference is we arent breathing heavy or useing most of the body like in the gym, so not as much whole body stress ... however and I have NO SCIENTIFIC PROOF here, just my opinion, when the penis has been exercised it too has dug into some of the bodies reserve energy and this can be higher with the more intensity you use in the session.
I THINK that it places more stress on the circulation, nerve and hormonal systems. You need the good circulation for the erection and indeed good nerve impulse for the erection, the horomones are self explanatory. I dont know how deep Penis Enlargement workouts would dig into the bodies system but they will dig in, it has to for the body to ADAPT to the STRESS and change to it, by this change you get new growth ... all that new penis growth was caused from STRESS placed on the body that it had to adapt to and when it was adapting to this, it was digging into the bodies reserves of energy ... if someone did Penis Enlargement for too long than this could cause problems such as overtraining, weak erections, poor sex drive, more injuries, lack of interest in doing the routines, loss of gains etc etc.
So the penis is treated like a muscle by guys like me not because of misconceptions about it being the same muscle type we work in the gym, but because its adaptation and recovery is the same as everything else in the human body ... it NEEDS chance to adapt to the stimulation placed upon it, to heal and recover from that than to grow after the healing has taken place.
Guys who exercise with low intensity wont need to take as many breaks imo because they arent digging too deep in the reserves to really warrant long periods of rest and sleep will aid them but in the longrun if someone does train day in and day out with no rest, they WILL hit a brickwall and if they dont and never hit a plateu than they are one off miracles to god, because it just wont happen ... the body will eventually need to stop it on its own, and hence the plateu kicks in for the body to tell you to back the fuck off.