Actions are guided by a combination of external cues, internal intentions and stored knowledge. Self-initiated voluntary actions, produced without any immediate external cue, may be preceded by a slow EEG Readiness Potential (RP) that progressively increases prior to action. The cognitive significance of this neural event is controversial. Some accounts link the RP to the fact that timing of voluntary actions is generated endogenously, without external constraints, and perhaps even randomly. Other accounts take the RP as reflecting the unique role of planning, therefore of temporal expectation, in voluntary actions. In many previous experiments, actions are both unconstrained by external cues, but also potentially involve preplanning and anticipation. To separate these factors, we developed a reinforcement learning paradigm where participants learned, through trial and error, the optimal time to act. If the RP reflects freedom from external constraint, its amplitude should be greater early in learning, when participants do not yet know the best time to act. Conversely, if the RP reflects planning, it should be greater later on, when participants have learned, and know in advance, the time of action. We found that RP amplitudes grew with learning, suggesting that this neural activity reflects planning and anticipation for the forthcoming action, rather than freedom from external constraint.