explain this code: + +#if ISIN (libc) +ENTRYALIGN (_memcpyfalkor, 6) + + cmp count, 32 +…
- User
explain this code:
+ +#if IS_IN (libc) +ENTRY_ALIGN (__memcpy_falkor, 6) +
- cmp count, 32
- add srcend, src, count
- add dstend, dstin, count
- b.ls L(copy32)
- ldp A_l, A_h, [src]
- cmp count, 128
- stp A_l, A_h, [dstin]
- b.hi L(copy_long)
- /* Medium copies: 33..128 bytes. */
- sub tmp1, count, 1
- ldp A_l, A_h, [src, 16]
- stp A_l, A_h, [dstin, 16]
- tbz tmp1, 6, 1f
- ldp A_l, A_h, [src, 32]
- stp A_l, A_h, [dstin, 32]
- ldp A_l, A_h, [src, 48]
- stp A_l, A_h, [dstin, 48]
- ldp A_l, A_h, [srcend, -64]
- stp A_l, A_h, [dstend, -64]
- ldp A_l, A_h, [srcend, -48]
- stp A_l, A_h, [dstend, -48] +1:
- ldp A_l, A_h, [srcend, -32]
- stp A_l, A_h, [dstend, -32]
- ldp A_l, A_h, [srcend, -16]
- stp A_l, A_h, [dstend, -16]
- ret
- .p2align 4
- /* Small copies: 0..32 bytes. */ +L(copy32):
- /* 16-32 */
- cmp count, 16
- b.lo 1f
- ldp A_l, A_h, [src]
- stp A_l, A_h, [dstin]
- ldp A_l, A_h, [srcend, -16]
- stp A_l, A_h, [dstend, -16]
- ret
- .p2align 4 +1:
- /* 8-15 */
- tbz count, 3, 1f
- ldr A_l, [src]
- str A_l, [dstin]
- ldr A_l, [srcend, -8]
- str A_l, [dstend, -8]
- ret
- .p2align 4 +1:
- /* 4-7 */
- tbz count, 2, 1f
- ldr A_lw, [src]
- str A_lw, [dstin]
- ldr A_lw, [srcend, -4]
- str A_lw, [dstend, -4]
- ret
- .p2align 4 +1:
- /* 2-3 */
- tbz count, 1, 1f
- ldrh A_lw, [src]
- strh A_lw, [dstin]
- ldrh A_lw, [srcend, -2]
- strh A_lw, [dstend, -2]
- ret
- .p2align 4 +1:
- /* 0-1 */
- tbz count, 0, 1f
- ldrb A_lw, [src]
- strb A_lw, [dstin] +1:
- ret
- /* Align SRC to 16 bytes and copy; that way at least one of the
- accesses is aligned throughout the copy sequence.
- The count is off by 0 to 15 bytes, but this is OK because we trim
- off the last 64 bytes to copy off from the end. Due to this the
- loop never runs out of bounds. */
- .p2align 6 +L(copy_long):
- sub count, count, 64 + 16
- and tmp1, src, 15
- bic src, src, 15
- sub dst, dstin, tmp1
- add count, count, tmp1
+L(loop64):
- ldp A_l, A_h, [src, 16]!
- stnp A_l, A_h, [dst, 16]
- ldp A_l, A_h, [src, 16]!
- subs count, count, 64
- stnp A_l, A_h, [dst, 32]
- ldp A_l, A_h, [src, 16]!
- stnp A_l, A_h, [dst, 48]
- ldp A_l, A_h, [src, 16]!
- stnp A_l, A_h, [dst, 64]
- add dst, dst, 64
- b.hi L(loop64)
- /* Write the last full set of 64 bytes. The remainder is at most 64
- bytes, so it is safe to always copy 64 bytes from the end even if
- there is just 1 byte left. */ +L(last64):
- ldp A_l, A_h, [srcend, -64]
- stnp A_l, A_h, [dstend, -64]
- ldp A_l, A_h, [srcend, -48]
- stnp A_l, A_h, [dstend, -48]
- ldp A_l, A_h, [srcend, -32]
- stnp A_l, A_h, [dstend, -32]
- ldp A_l, A_h, [srcend, -16]
- stnp A_l, A_h, [dstend, -16]
- ret
+END (__memcpy_falkor) +libc_hidden_builtin_def (__memcpy_falkor) +#endif
- Assistant
The provided code is an optimized implementation of the
memcpyfunction, specifically tailored for the ARM architecture (as indicated by the usage of ARM assembly instructions likeldp,stp,tbz, etc.). The function is named__memcpy_falkor, suggesting it may be intended for a specific platform or processor variant codenamed "Falkor". Let's dissect the code step-by-step to understand its functionality and optimizations.
1. Preprocessor Directives and Function Entry
#if IS_IN (libc) ENTRY_ALIGN (__memcpy_falkor, 6)#if IS_IN (libc): This line checks if the function is part of the C standard library (libc).ENTRY_ALIGN (__memcpy_falkor, 6): Aligns the entry point of the__memcpy_falkorfunction to a 64-byte boundary (2^6), which can enhance performance by ensuring proper alignment for cache lines.
2. Initial Setup and Branching Based on
countcmp count, 32 add srcend, src, count add dstend, dstin, count b.ls L(copy32) ldp A_l, A_h, [src] cmp count, 128 stp A_l, A_h, [dstin] b.hi L(copy_long)cmp count, 32: Compares the number of bytes to copy (count) with 32.add srcend, src, countandadd dstend, dstin, count: Calculate the end addresses for the source (srcend) and destination (dstend).b.ls L(copy32): Ifcountis less than or equal to 32, branch to thecopy32label to handle small copies.ldp A_l, A_h, [src]: Loads a pair of 64-bit values from the source into registersA_landA_h.cmp count, 128: Comparescountwith 128.stp A_l, A_h, [dstin]: Stores the loaded pair into the destination.b.hi L(copy_long): Ifcountis greater than 128, branch to thecopy_longlabel to handle large copies.
3. Handling Medium-Sized Copies (33–128 Bytes)
/* Medium copies: 33..128 bytes. */ sub tmp1, count, 1 ldp A_l, A_h, [src, 16] stp A_l, A_h, [dstin, 16] ldp A_l, A_h, [src, 32] stp A_l, A_h, [dstin, 32] ldp A_l, A_h, [src, 48] stp A_l, A_h, [dstin, 48] ldp A_l, A_h, [srcend, -64] stp A_l, A_h, [dstend, -64] ldp A_l, A_h, [srcend, -48] stp A_l, A_h, [dstend, -48] 1: ldp A_l, A_h, [srcend, -32] stp A_l, A_h, [dstend, -32] ldp A_l, A_h, [srcend, -16] stp A_l, A_h, [dstend, -16] retsub tmp1, count, 1: Adjusts the count, possibly for alignment or boundary conditions.- Loading and Storing: The function loads multiple pairs (
A_l,A_h) from the source at various offsets (16, 32, 48 bytes) and stores them into the destination at corresponding offsets. - Handling the End Part: It also handles the end of the memory region by loading from
srcendwith negative offsets and storing todstend. - Final Steps: After copying the necessary chunks, the function returns.
4. Handling Small Copies (0–32 Bytes)
.p2align 4 /* Small copies: 0..32 bytes. */ L(copy32): /* 16-32 */ cmp count, 16 b.lo 1f ldp A_l, A_h, [src] stp A_l, A_h, [dstin] ldp A_l, A_h, [srcend, -16] stp A_l, A_h, [dstend, -16] ret .p2align 4 1: /* 8-15 */ tbz count, 3, 1f ldr A_l, [src] str A_l, [dstin] ldr A_l, [srcend, -8] str A_l, [dstend, -8] ret .p2align 4 1: /* 4-7 */ tbz count, 2, 1f ldr A_lw, [src] str A_lw, [dstin] ldr A_lw, [srcend, -4] str A_lw, [dstend, -4] ret .p2align 4 1: /* 2-3 */ tbz count, 1, 1f ldrh A_lw, [src] strh A_lw, [dstin] ldrh A_lw, [srcend, -2] strh A_lw, [dstend, -2] ret .p2align 4 1: /* 0-1 */ tbz count, 0, 1f ldrb A_lw, [src] strb A_lw, [dstin] 1: retThe small copy section is further broken down based on the exact number of bytes to copy:
-
16–32 Bytes:
cmp count, 16: Checks ifcountis greater than 16.- If
count >= 16, it copies 16 bytes fromsrctodstinand another 16 bytes fromsrcend - 16todstend - 16. ret: Returns after copying.
-
8–15 Bytes:
tbz count, 3, 1f: Tests bit 3 ofcount. If not set, branches to label1fto handle smaller sizes.- If
count >= 8, it copies 8 bytes and another 8 bytes from the end. ret: Returns after copying.
-
4–7 Bytes:
tbz count, 2, 1f: Tests bit 2 ofcount.- If
count >= 4, it copies 4 bytes and another 4 bytes from the end. ret: Returns after copying.
-
2–3 Bytes:
tbz count, 1, 1f: Tests bit 1 ofcount.- If
count >= 2, it copies 2 bytes and another 2 bytes from the end. ret: Returns after copying.
-
0–1 Bytes:
tbz count, 0, 1f: Tests bit 0 ofcount.- If
count >= 1, it copies 1 byte. ret: Returns after copying.
This hierarchical approach ensures that the function handles all possible small
countvalues efficiently by copying the exact number of required bytes without overstepping memory boundaries.
5. Handling Large Copies (>128 Bytes)
/* Align SRC to 16 bytes and copy; that way at least one of the accesses is aligned throughout the copy sequence. The count is off by 0 to 15 bytes, but this is OK because we trim off the last 64 bytes to copy off from the end. Due to this the loop never runs out of bounds. */ .p2align 6 L(copy_long): sub count, count, 64 + 16 and tmp1, src, 15 bic src, src, 15 sub dst, dstin, tmp1 add count, count, tmp1 L(loop64): ldp A_l, A_h, [src, 16]! stnp A_l, A_h, [dst, 16] ldp A_l, A_h, [src, 16]! subs count, count, 64 stnp A_l, A_h, [dst, 32] ldp A_l, A_h, [src, 16]! stnp A_l, A_h, [dst, 48] ldp A_l, A_h, [src, 16]! stnp A_l, A_h, [dst, 64] add dst, dst, 64 b.hi L(loop64)For large
countvalues (greater than 128 bytes), the function employs a loop to efficiently copy large blocks of memory:-
Alignment:
and tmp1, src, 15: Calculates the misalignment of thesrcpointer relative to a 16-byte boundary.bic src, src, 15: Aligns thesrcpointer down to the nearest 16-byte boundary.sub dst, dstin, tmp1andadd count, count, tmp1: Adjust thedstpointer and thecountto account for this alignment adjustment.
-
Loop (
L(loop64)):ldpandstnpInstructions: These load and store multiple pairs of 64-bit data efficiently.ldp A_l, A_h, [src, 16]!: Loads two 64-bit values fromsrcand incrementssrcby 16 bytes.stnp A_l, A_h, [dst, 16]: Stores the loaded pair todstwith an offset.
- Iterations:
- The loop performs multiple load-store operations per iteration, handling 64 bytes of data each time.
subs count, count, 64: Decrements thecountby 64 bytes and updates condition flags.b.hi L(loop64): If there are more bytes to copy (count > 0), the loop continues.
This loop efficiently handles large memory regions by minimizing the number of instructions and maximizing data throughput, leveraging ARM's parallel load-store capabilities.
6. Finalizing Large Copies
/* Write the last full set of 64 bytes. The remainder is at most 64 bytes, so it is safe to always copy 64 bytes from the end even if there is just 1 byte left. */ L(last64): ldp A_l, A_h, [srcend, -64] stnp A_l, A_h, [dstend, -64] ldp A_l, A_h, [srcend, -48] stnp A_l, A_h, [dstend, -48] ldp A_l, A_h, [srcend, -32] stnp A_l, A_h, [dstend, -32] ldp A_l, A_h, [srcend, -16] stnp A_l, A_h, [dstend, -16] ret- Safety Mechanism: After the loop, there might still be up to 64 bytes left to copy. This section ensures that the last chunk of data is copied without overrunning memory boundaries.
- Loading and Storing: It methodically copies four more 16-byte blocks from the end of the
srcto thedst. ret: Returns after the final copying step.
7. Function End and Visibility
END (__memcpy_falkor) libc_hidden_builtin_def (__memcpy_falkor) #endifEND (__memcpy_falkor): Marks the end of the__memcpy_falkorfunction.libc_hidden_builtin_def (__memcpy_falkor): This macro likely ensures that__memcpy_falkoris treated as a hidden built-in function within thelibc, affecting its symbol visibility and linkage.
Summary
The
__memcpy_falkorfunction is a highly optimized implementation of the standardmemcpyfunction, tailored for the ARM architecture. It employs several optimization strategies:- Alignment: Ensures that memory accesses are aligned to 16-byte boundaries to enhance performance and leverage ARM's efficient load/store capabilities.
- Conditional Branching: Divides the copying process based on the size (
count) into small (0–32 bytes), medium (33–128 bytes), and large (>128 bytes) copies, optimizing each scenario with tailored strategies. - Loop Unrolling: For large copies, it unrolls the loop to handle multiple blocks per iteration, reducing the overhead of loop control and maximizing data throughput.
- Efficient Load/Store Operations: Utilizes ARM's
ldpandstpinstructions for loading and storing pairs of registers, andstnpfor storing non-post-indexed pairs, which are more efficient than individual load/store operations. - End Handling: Carefully manages the end of the memory region to prevent overruns and ensure that all bytes are copied correctly, regardless of alignment.
By meticulously handling different copy sizes and leveraging ARM's architectural features,
__memcpy_falkorachieves high performance and reliability for memory copy operations.