After disabling log-bin by skip-log-bin option, I cannot use PURGE command to remove binary logs.
Refer to the documentation
PURGE BINARY LOGS requires the BINLOG_ADMIN privilege. This statement has no effect if the server was not started with the --log-bin option to enable binary logging.
My question is: If I need reenable log-bin in the future, is it safe to delete remaining binary logs with rm binlog.* command now?