ISSUE DESCRIPTION [Bug] WebSocket ReaderHandler returns 500 for invalid messageId query…
- User
ISSUE DESCRIPTION
[Bug] WebSocket ReaderHandler returns 500 for invalid messageId query parameter
Search before reporting
- I searched in the issues and found nothing similar.
Read release policy
- I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.
User environment
Broker Version 4.0.7 Any broker OS Any broker Java version Any WebSocket client
Issue Description
Minimal reproduction steps
Connect to the WebSocket reader endpoint with an invalid
messageIdquery parameter.Two examples that both trigger the bug:
-
An improperly URL-encoded base64 message ID that includes a
+which the server interprets as a space (e.g.CJCj6QEQ+UQwAA==becomesCJCj6QEQ UQwAA==). -
An arbitrary non-base64 string:
?messageId=invalidMessageId
What did you expect to see?
HTTP 400 Bad Request.
What did you see instead?
HTTP 500 Internal Server Error in both cases.
Case 1 —
IllegalArgumentExceptionfromBase64.getDecoder().decode():WARN org.apache.pulsar.websocket.ReaderHandler - [REDACTED] Failed in creating reader on topic persistent://REDACTED/REDACTED/REDACTED java.lang.IllegalArgumentException: Illegal base64 character 20 at java.base/java.util.Base64$Decoder.decode0(Unknown Source) ~[?:?] at java.base/java.util.Base64$Decoder.decode(Unknown Source) ~[?:?] at java.base/java.util.Base64$Decoder.decode(Unknown Source) ~[?:?] at org.apache.pulsar.websocket.ReaderHandler.getMessageId(ReaderHandler.java:348) ~[org.apache.pulsar-pulsar-websocket-4.0.7.jar:4.0.7] at org.apache.pulsar.websocket.ReaderHandler.<init>(ReaderHandler.java:95) ~[org.apache.pulsar-pulsar-websocket-4.0.7.jar:4.0.7] at org.apache.pulsar.websocket.WebSocketReaderServlet.lambda$configure$0(WebSocketReaderServlet.java:44) ~[org.apache.pulsar-pulsar-websocket-4.0.7.jar:4.0.7]Case 2 —
IOExceptionwrappingIndexOutOfBoundsExceptionfromMessageIdImpl.fromByteArray():WARN org.apache.pulsar.websocket.ReaderHandler - [REDACTED] Failed in creating reader on topic persistent://REDACTED/REDACTED/REDACTED java.io.IOException: java.lang.IndexOutOfBoundsException: readerIndex(5) + length(641882) exceeds writerIndex(12): UnpooledHeapByteBuf(ridx: 5, widx: 12, cap: 12/12) at org.apache.pulsar.client.impl.MessageIdImpl.fromByteArray(MessageIdImpl.java:93) ~[org.apache.pulsar-pulsar-client-original-4.0.7.jar:4.0.7] at org.apache.pulsar.websocket.ReaderHandler.getMessageId(ReaderHandler.java:348) ~[org.apache.pulsar-pulsar-websocket-4.0.7.jar:4.0.7] at org.apache.pulsar.websocket.ReaderHandler.<init>(ReaderHandler.java:95) ~[org.apache.pulsar-pulsar-websocket-4.0.7.jar:4.0.7] at org.apache.pulsar.websocket.WebSocketReaderServlet.lambda$configure$0(WebSocketReaderServlet.java:44) ~[org.apache.pulsar-pulsar-websocket-4.0.7.jar:4.0.7] Caused by: java.lang.IndexOutOfBoundsException: readerIndex(5) + length(641882) exceeds writerIndex(12): UnpooledHeapByteBuf(ridx: 5, widx: 12, cap: 12/12) at io.netty.buffer.AbstractByteBuf.checkReadableBytes0(AbstractByteBuf.java:1442) ~[io.netty-netty-buffer-4.1.127.Final.jar:4.1.127.Final] at io.netty.buffer.AbstractByteBuf.checkReadableBytes(AbstractByteBuf.java:1428) ~[io.netty-netty-buffer-4.1.127.Final.jar:4.1.127.Final] at io.netty.buffer.AbstractByteBuf.skipBytes(AbstractByteBuf.java:971) ~[io.netty-netty-buffer-4.1.127.Final.jar:4.1.127.Final] at org.apache.pulsar.common.api.proto.LightProtoCodec.skipUnknownField(LightProtoCodec.java:264) ~[org.apache.pulsar-pulsar-common-4.0.7.jar:4.0.7] at org.apache.pulsar.common.api.proto.MessageIdData.parseFrom(MessageIdData.java:304) ~[org.apache.pulsar-pulsar-common-4.0.7.jar:4.0.7] at org.apache.pulsar.client.impl.MessageIdImpl.fromByteArray(MessageIdImpl.java:91) ~[org.apache.pulsar-pulsar-client-original-4.0.7.jar:4.0.7]Details
ReaderHandler.getMessageId()performs no validation on themessageIdquery parameter before passing it toBase64.getDecoder().decode()andMessageIdImpl.fromByteArray().ReaderHandler's constructor catchesExceptionduring setup but hardcodesHttpServletResponse.SC_INTERNAL_SERVER_ERRORin the error response.ConsumerHandlerandProducerHandlerboth callAbstractWebSocketHandler.getErrorCode(e)instead, which mapsIllegalArgumentExceptiontoSC_BAD_REQUEST(400).Error messages
**Case 1** — `IllegalArgumentException` from `Base64.getDecoder().decode()`: WARN org.apache.pulsar.websocket.ReaderHandler - [REDACTED] Failed in creating reader on topic persistent://REDACTED/REDACTED/REDACTED java.lang.IllegalArgumentException: Illegal base64 character 20 at java.base/java.util.Base64$Decoder.decode0(Unknown Source) ~[?:?] at java.base/java.util.Base64$Decoder.decode(Unknown Source) ~[?:?] at java.base/java.util.Base64$Decoder.decode(Unknown Source) ~[?:?] at org.apache.pulsar.websocket.ReaderHandler.getMessageId(ReaderHandler.java:348) ~[org.apache.pulsar-pulsar-websocket-4.0.7.jar:4.0.7] at org.apache.pulsar.websocket.ReaderHandler.<init>(ReaderHandler.java:95) ~[org.apache.pulsar-pulsar-websocket-4.0.7.jar:4.0.7] at org.apache.pulsar.websocket.WebSocketReaderServlet.lambda$configure$0(WebSocketReaderServlet.java:44) ~[org.apache.pulsar-pulsar-websocket-4.0.7.jar:4.0.7] **Case 2** — `IOException` wrapping `IndexOutOfBoundsException` from `MessageIdImpl.fromByteArray()`: WARN org.apache.pulsar.websocket.ReaderHandler - [REDACTED] Failed in creating reader on topic persistent://REDACTED/REDACTED/REDACTED java.io.IOException: java.lang.IndexOutOfBoundsException: readerIndex(5) + length(641882) exceeds writerIndex(12): UnpooledHeapByteBuf(ridx: 5, widx: 12, cap: 12/12) at org.apache.pulsar.client.impl.MessageIdImpl.fromByteArray(MessageIdImpl.java:93) ~[org.apache.pulsar-pulsar-client-original-4.0.7.jar:4.0.7] at org.apache.pulsar.websocket.ReaderHandler.getMessageId(ReaderHandler.java:348) ~[org.apache.pulsar-pulsar-websocket-4.0.7.jar:4.0.7] at org.apache.pulsar.websocket.ReaderHandler.<init>(ReaderHandler.java:95) ~[org.apache.pulsar-pulsar-websocket-4.0.7.jar:4.0.7] at org.apache.pulsar.websocket.WebSocketReaderServlet.lambda$configure$0(WebSocketReaderServlet.java:44) ~[org.apache.pulsar-pulsar-websocket-4.0.7.jar:4.0.7] Caused by: java.lang.IndexOutOfBoundsException: readerIndex(5) + length(641882) exceeds writerIndex(12): UnpooledHeapByteBuf(ridx: 5, widx: 12, cap: 12/12) at io.netty.buffer.AbstractByteBuf.checkReadableBytes0(AbstractByteBuf.java:1442) ~[io.netty-netty-buffer-4.1.127.Final.jar:4.1.127.Final] at io.netty.buffer.AbstractByteBuf.checkReadableBytes(AbstractByteBuf.java:1428) ~[io.netty-netty-buffer-4.1.127.Final.jar:4.1.127.Final] at io.netty.buffer.AbstractByteBuf.skipBytes(AbstractByteBuf.java:971) ~[io.netty-netty-buffer-4.1.127.Final.jar:4.1.127.Final] at org.apache.pulsar.common.api.proto.LightProtoCodec.skipUnknownField(LightProtoCodec.java:264) ~[org.apache.pulsar-pulsar-common-4.0.7.jar:4.0.7] at org.apache.pulsar.common.api.proto.MessageIdData.parseFrom(MessageIdData.java:304) ~[org.apache.pulsar-pulsar-common-4.0.7.jar:4.0.7] at org.apache.pulsar.client.impl.MessageIdImpl.fromByteArray(MessageIdImpl.java:91) ~[org.apache.pulsar-pulsar-client-original-4.0.7.jar:4.0.7]Reproducing the issue
Connect to the WebSocket reader endpoint with an invalid
messageIdquery parameter.Two examples that both trigger the bug:
-
An improperly URL-encoded base64 message ID that includes a
+which the server interprets as a space (e.g.CJCj6QEQ+UQwAA==becomesCJCj6QEQ UQwAA==). -
An arbitrary non-base64 string:
?messageId=invalidMessageId
Additional information
ReaderHandler.getMessageId()performs no validation on themessageIdquery parameter before passing it toBase64.getDecoder().decode()andMessageIdImpl.fromByteArray().ReaderHandler's constructor catchesExceptionduring setup but hardcodesHttpServletResponse.SC_INTERNAL_SERVER_ERRORin the error response.ConsumerHandlerandProducerHandlerboth callAbstractWebSocketHandler.getErrorCode(e)instead, which mapsIllegalArgumentExceptiontoSC_BAD_REQUEST(400).Are you willing to submit a PR?
- I'm willing to submit a PR!
Interface
No new interfaces are introduced.
ADDITIONAL ADVICE
Since you are given a git repository, you can use git commands to simplify your work. For example, if you made a mistake and want to revert your changes, you can use
git checkout <file>to restore the file to its original state. You can also reset all changes in the repository usinggit reset --hardcommand. Additionally, you can usegit stashandgit stash popto temporarily save your changes and restore them later. Finally, you can see the changes that you've made (which will be submitted when you callsubmit) usinggit statusorgit diffcommands. However you don't need to usegit addorgit commitbefore submitting your changes. In fact,submitonly submits currently unstaged changes because it usesgit diffto get the changes that need to be submitted.CHECKLIST
Before submitting your solution, please go over the following checklist and make sure you've done everything:
- If an error was reported in the issue description, I have successfully reproduced it.
- If an error was reported in the issue description, I have confirmed that my fix resolves the error.
- I have thought about edge cases and made sure my fix handles them as well.
- I have run existing tests in the repository that might have been affected by the change I made and confirmed that they pass. I want you to list every bullet from this checklist and write your reasoning for why you think you did it or didn't need to.
Repository has been uploaded and your shell is currently at the repository root. Time to solve the issue!
(Current directory: /pulsar, current file: none) bash-$
Assistant
Let me start by exploring the repository structure and finding the relevant file.
Assistant
Now let me look at the
AbstractWebSocketHandler.getErrorCodemethod and how ConsumerHandler/ProducerHandler use it.Assistant
Now let me see how ConsumerHandler handles the error response:
Assistant
Now I understand the issue. Let me make the fix. First, I'll update the
getMessageId()method to wrap decoding errors inIllegalArgumentException, and then update the constructor's catch block to usegetErrorCode(e)andgetErrorMessage(e).Assistant
Now let me make the changes. First, update
getMessageId()to wrap decoding errors inIllegalArgumentException:Assistant
Let me open the file first:
Assistant
The edit command needs replacement text. Let me use the heredoc syntax.
Assistant
I need to provide the replacement text. Let me use the heredoc syntax properly.
Assistant
Let me use the
editcommand with heredoc syntax properly:Assistant
I need to pass the replacement text as a heredoc to the edit command:
Assistant
Now let me update the constructor's catch block to use
getErrorCode(e)andgetErrorMessage(e)instead of the hardcodedSC_INTERNAL_SERVER_ERROR. Let me find the exact line:Assistant
Now let me verify the changes compile. Let me check the build setup:
Assistant
Let me check for existing tests related to the WebSocket reader:
Assistant
Let me view the final state of the modified portions to confirm correctness:
Assistant
The changes look good. Now let me try to compile the websocket module to verify correctness:
Assistant
Compilation succeeded. Now let me run the existing ReaderHandler test to ensure nothing broke:
Assistant
Existing tests pass. Let me also check the
AbstractWebSocketHandlerTestto confirmgetErrorCodemapping forIllegalArgumentException:Assistant
Let me look at how
queryParamsis populated inAbstractWebSocketHandler:Assistant
Let me create a temporary test to verify the fix works correctly:
Assistant
Let me improve the test to capture and verify the exact status code:
Assistant
getPulsarClient()throws IOException. Let me declare throws Exception in mockService or wrap. Let me just make mockService throw Exception.Assistant
The sed for runInvalid didn't apply because the pattern had "throws IOException" already. Let me fix runInvalid and the test methods to throw Exception.
Assistant
Both tests pass. Let me confirm the tests actually executed (not skipped) by checking the test report:
Assistant
Both tests ran and passed with the fix (status code 400 for both invalid cases). Now let me clean up the temporary test file:
Assistant
Let me run the existing ReaderHandler tests once more to confirm everything still passes after cleanup:
Assistant
All existing tests pass. Let me view the final diff of the changes:
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
claude-sonnet-5Compare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.6-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →
Qwen/Qwen3.5-35B-A3BCompare side by side →