The plastic in the shower walls will expand and contract at different rates than the drywall, so it will crack - silicon will stretch some to prevent that from happening. The shower door or curtain should keep moisture away from that edge, but you may not like the crack that eventually is highly likely to show up. If the expansion is more than expected, having a rigid joint might put more stress on the panels. I doubt they'd generate stress cracks, but I suppose they could. It's always best to follow the manufacturer's instructions.