LLVM/project dfb661cllvm/test/Analysis/LoopAccessAnalysis early-exit-runtime-checks.ll evaluate-at-symbolic-max-backedge-taken-count-may-wrap.ll, llvm/test/Transforms/LoopVectorize single_early_exit_live_outs.ll

[LAA] Add extra tests for #128061.

Extend test coverage for
https://github.com/llvm/llvm-project/pull/128061.
DeltaFile
+186-18llvm/test/Analysis/LoopAccessAnalysis/early-exit-runtime-checks.ll
+57-8llvm/test/Transforms/LoopVectorize/single_early_exit_live_outs.ll
+43-0llvm/test/Analysis/LoopAccessAnalysis/evaluate-at-symbolic-max-backedge-taken-count-may-wrap.ll
+286-263 files

LLVM/project f13d583llvm/lib/Transforms/Vectorize VPlanUtils.cpp

[VPlan] Pass some functions directly to all_of (NFC).

Remove some unneeded lambdas.
DeltaFile
+2-4llvm/lib/Transforms/Vectorize/VPlanUtils.cpp
+2-41 files

LLVM/project 02575f8llvm/lib/Transforms/Vectorize VPlanRecipes.cpp VPlan.h

[VPlan] Use VPInstruction for VPScalarPHIRecipe. (NFCI) (#129767)

Now that all phi nodes manage their incoming blocks through the
VPlan-predecessors, there should be no need for having a dedicate
recipe, it should be sufficient to allow PHI opcodes in VPInstruction.

Follow-ups will also migrate VPWidenPHIRecipe and possibly others,
building on top of https://github.com/llvm/llvm-project/pull/129388.

PR: https://github.com/llvm/llvm-project/pull/129767
DeltaFile
+28-25llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
+1-42llvm/lib/Transforms/Vectorize/VPlan.h
+12-8llvm/lib/Transforms/Vectorize/VPlanAnalysis.cpp
+9-4llvm/lib/Transforms/Vectorize/VPlanVerifier.cpp
+6-5llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
+5-3llvm/lib/Transforms/Vectorize/VPlan.cpp
+61-876 files not shown
+69-9512 files

LLVM/project 62994c3llvm/lib/Transforms/Vectorize VPlanTransforms.cpp, llvm/test/Transforms/LoopVectorize expand-scev-after-invoke.ll induction-step.ll

[VPlan] Also introduce explicit broadcasts for values from entry VPBB.

Update and generalize materializeBroadcasts to also introduce explicit
broadcasts for VPValues defined in the Plans Entry block.

This fixes a crash when trying to insert the broadcasts generated by
VPTransformState::get after the generating instruction, which isn't
possible after invoke instructions.

Fixes https://github.com/llvm/llvm-project/issues/128838.
DeltaFile
+83-0llvm/test/Transforms/LoopVectorize/expand-scev-after-invoke.ll
+19-12llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
+8-8llvm/test/Transforms/LoopVectorize/X86/epilog-vectorization-inductions.ll
+8-8llvm/test/Transforms/LoopVectorize/X86/induction-step.ll
+6-6llvm/test/Transforms/LoopVectorize/induction-step.ll
+5-5llvm/test/Transforms/LoopVectorize/float-induction.ll
+129-394 files not shown
+139-4910 files

LLVM/project dc23234llvm/lib/Transforms/Vectorize VPlanRecipes.cpp

[VPlan] Remove dead code in VPWidenPHIRecipe::print (NFC).

All incoming models for VPWidenPHIRecipe are modled in VPlan, remove
code trying to print the orignial phi.
DeltaFile
+0-10llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
+0-101 files

LLVM/project 8132c4fllvm/lib/Transforms/Vectorize VPlanTransforms.cpp, llvm/test/Transforms/LoopVectorize pr37248.ll pr55167-fold-tail-live-out.ll

[VPlan] Also introduce broadcasts for live-ins used in vec preheader.

Slightly generalize materializeLiveInBroadcasts to also introduce
broadcasts for live-ins used in the vector preheader. This should cover
all live-ins.

If the live-in is used in the vector preheader, insert the broadcast at
the beginning of the block.
DeltaFile
+35-35llvm/test/Transforms/LoopVectorize/pr37248.ll
+21-17llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
+8-8llvm/test/Transforms/LoopVectorize/AArch64/reduction-recurrence-costs-sve.ll
+7-7llvm/test/Transforms/LoopVectorize/pr55167-fold-tail-live-out.ll
+4-4llvm/test/Transforms/LoopVectorize/X86/scatter_crash.ll
+4-4llvm/test/Transforms/LoopVectorize/AArch64/loop-vectorization-factors.ll
+79-7513 files not shown
+101-9719 files

LLVM/project aeae366llvm/lib/Transforms/Vectorize VPlan.h

[VPlan] Make start operand non-optional for VPHeaderPHIRecipe (NFC).

The start value is always available, require it unconditionally.
DeltaFile
+2-4llvm/lib/Transforms/Vectorize/VPlan.h
+2-41 files

LLVM/project f84f4e1llvm/lib/Transforms/Vectorize LoopVectorize.cpp, llvm/test/Transforms/LoopVectorize vplan-native-path-inner-loop-with-runtime-checks.ll

[LV] Don't crash on inner loops with RT checks in VPlan-native path.

Assert that we only generate runtime checks for inner loops in
emitMemRuntimeChecks, instead of returning nullptr in the VPlan-native
path, which is causing crashes and incorrect code.
DeltaFile
+157-0llvm/test/Transforms/LoopVectorize/vplan-native-path-inner-loop-with-runtime-checks.ll
+4-4llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+161-42 files

LLVM/project c7f7ac7llvm/lib/Transforms/IPO MergeFunctions.cpp, llvm/test/Transforms/MergeFunc comdat.ll

[MergeFunc] Keep comdat on new function, not thunk. (#130583)

MergeFunc uses the original function F as Thunk and creates a new function NewF for the original function F. Preserve F's comdat info on NewF instead of the thunk.

This fixes a crash when trying to lower comdat on the thunk during codegen.
DeltaFile
+6-4llvm/test/Transforms/MergeFunc/comdat.ll
+2-0llvm/lib/Transforms/IPO/MergeFunctions.cpp
+8-42 files

LLVM/project dd86ecellvm/test/Transforms/MergeFunc comdat.ll

[MergeFunc] Check full IR and comdat keys in comdat.ll.
DeltaFile
+24-3llvm/test/Transforms/MergeFunc/comdat.ll
+24-31 files

LLVM/project 437d587llvm/test/Transforms/LoopVectorize outer-loop-inner-latch-successors.ll

[LV] Add outer loop test with different successor orders in inner latch.
DeltaFile
+213-0llvm/test/Transforms/LoopVectorize/outer-loop-inner-latch-successors.ll
+213-01 files

LLVM/project fd26708llvm/lib/Transforms/Vectorize VPlanConstruction.cpp VPlan.cpp, llvm/test/Transforms/LoopVectorize vplan-printing-outer-loop.ll

[VPlan] Refactor VPlan creation, add transform introducing region (NFC). (#128419)

Create an empty VPlan first, then let the HCFG builder create a plain
CFG for the top-level loop (w/o a top-level region). The top-level
region is introduced by a separate VPlan-transform. This is instead of
creating the vector loop region before building the VPlan CFG for the
input loop.

This simplifies the HCFG builder (which should probably be renamed) and
moves along the roadmap ('buildLoop') outlined in [1].

As follow-up, I plan to also preserve the exit branches in the initial
VPlan out of the CFG builder, including connections to the exit blocks.

The conversion from plain CFG with potentially multiple exits to a
single entry/exit region will be done as VPlan transform in a follow-up.

This is needed to enable VPlan-based predication. Currently early exit
support relies on building the block-in masks on the original CFG,

    [13 lines not shown]
DeltaFile
+101-0llvm/lib/Transforms/Vectorize/VPlanConstruction.cpp
+7-84llvm/lib/Transforms/Vectorize/VPlan.cpp
+21-50llvm/test/Transforms/LoopVectorize/vplan-printing-outer-loop.ll
+21-44llvm/lib/Transforms/Vectorize/VPlanHCFGBuilder.cpp
+8-16llvm/lib/Transforms/Vectorize/VPlan.h
+11-7llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+169-2014 files not shown
+193-20710 files

LLVM/project 8dd160fllvm/test/Transforms/LoopVectorize select-cmp.ll single_early_exit_live_outs.ll, llvm/test/Transforms/LoopVectorize/AArch64 sve-select-cmp.ll

Revert "[VPlan] Fold NOT into predicate of wide compares." (#130347)

Reverts llvm/llvm-project#129430

this seems to have introduced a divergence between legacy and
VPlan-based cost model

https://lab.llvm.org/buildbot/#/builders/30/builds/17159
DeltaFile
+120-60llvm/test/Transforms/LoopVectorize/X86/masked_load_store.ll
+44-22llvm/test/Transforms/LoopVectorize/select-cmp.ll
+36-18llvm/test/Transforms/LoopVectorize/single_early_exit_live_outs.ll
+27-24llvm/test/Transforms/LoopVectorize/select-reduction-start-value-may-be-undef-or-poison.ll
+30-20llvm/test/Transforms/LoopVectorize/RISCV/select-cmp-reduction.ll
+21-14llvm/test/Transforms/LoopVectorize/AArch64/sve-select-cmp.ll
+278-15822 files not shown
+381-24128 files

LLVM/project cb3ce30llvm/test/Transforms/LoopVectorize select-cmp.ll single_early_exit_live_outs.ll, llvm/test/Transforms/LoopVectorize/AArch64 sve-select-cmp.ll

[VPlan] Fold NOT into predicate of wide compares. (#129430)

Add simplification to fold negation into a compare, if the negation is
the only user of the compare. This removes a number of redundant
negations.

Alive2 Proofs for FPCMP test changes:  https://alive2.llvm.org/ce/z/WGDz9U

PR: https://github.com/llvm/llvm-project/pull/129430
DeltaFile
+60-120llvm/test/Transforms/LoopVectorize/X86/masked_load_store.ll
+22-44llvm/test/Transforms/LoopVectorize/select-cmp.ll
+18-36llvm/test/Transforms/LoopVectorize/single_early_exit_live_outs.ll
+24-27llvm/test/Transforms/LoopVectorize/select-reduction-start-value-may-be-undef-or-poison.ll
+20-30llvm/test/Transforms/LoopVectorize/RISCV/select-cmp-reduction.ll
+14-21llvm/test/Transforms/LoopVectorize/AArch64/sve-select-cmp.ll
+158-27822 files not shown
+241-38128 files

LLVM/project 72376e1llvm/lib/Transforms/Vectorize VPlan.h

[VPlan] Remove unused VPWidenIntrinsicRecipe constructor (NFC)
DeltaFile
+0-6llvm/lib/Transforms/Vectorize/VPlan.h
+0-61 files

LLVM/project b2d70e8llvm/lib/Transforms/Vectorize VPlanTransforms.cpp

[VPlan] Use Builder to create cast recipes in VPlanTransforms (NFC).

Use VPBuilder in a few more places. This avoids manual insertions and
will make changing the cast recipe easier in the future.
DeltaFile
+9-11llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
+9-111 files

LLVM/project 15770a1llvm/lib/Transforms/Vectorize VPlanTransforms.cpp

[VPlan] Remove dead recipes in entry when merging regions. (NFC)

Also remove recipes in the entry of the region that will be removed.
This makes sure we don't leave any dead users around. NFC at the moment,
but avoids causing issues in the future.
DeltaFile
+5-0llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
+5-01 files

LLVM/project dfc5f37llvm/lib/Transforms/Vectorize VPlan.h

[VPlan] Move onlyFirstLaneUsed to VPWidenInductionRecipe (NFC).

Move onlyFirstLaneUsed from VPWidenIntOrFpInductionRecipe and
VPWidenPointerInduction to VPWidenInductionRecipe. Also mark step value
as having only its first lane used.
DeltaFile
+10-17llvm/lib/Transforms/Vectorize/VPlan.h
+10-171 files

LLVM/project 87f837cllvm/lib/Transforms/Vectorize VPlan.h

[VPlan] Remove unneeded classof with VPHeaderRecipe args (NFC).

The extra classof implementation is not needed any longer.
DeltaFile
+0-20llvm/lib/Transforms/Vectorize/VPlan.h
+0-201 files

LLVM/project ba7e273llvm/lib/Transforms/Vectorize VPlan.h

[VPlan] Use VP_CLASSOF_IMPL in VPWidenRecipe. (NFC)
DeltaFile
+1-8llvm/lib/Transforms/Vectorize/VPlan.h
+1-81 files

LLVM/project f937b17llvm/lib/Transforms/Vectorize LoopVectorize.cpp, llvm/test/Transforms/LoopVectorize/X86 cost-model.ll

[LV] Don't query SCEV for non-invariant values in cost model.

This fixes a divergence between VPlan and legacy cost model, matching
behavior further up in getInstructionCost as well.

Fixes https://github.com/llvm/llvm-project/issues/129236.
DeltaFile
+65-0llvm/test/Transforms/LoopVectorize/X86/cost-model.ll
+2-1llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+67-12 files

LLVM/project 75270e3llvm/lib/Transforms/Vectorize VPlanHCFGBuilder.h VPlanHCFGBuilder.cpp, llvm/unittests/Transforms/Vectorize VPlanVerifierTest.cpp

[VPlan] Don't print VPlan DT after VPlan construction. (NFC)

Remove unnecessary code to just print VPlan dominator tree.
DeltaFile
+3-6llvm/lib/Transforms/Vectorize/VPlanHCFGBuilder.h
+0-5llvm/lib/Transforms/Vectorize/VPlanHCFGBuilder.cpp
+1-0llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
+1-0llvm/unittests/Transforms/Vectorize/VPlanVerifierTest.cpp
+5-114 files

LLVM/project 9f37cdcllvm/lib/Transforms/Vectorize VPlanHelpers.h VPlan.cpp

[VPlan] Update VPTransformState accessors to take const VPValue (NFC).

This will enable using const VPValue * pointers are in more places.
DeltaFile
+13-11llvm/lib/Transforms/Vectorize/VPlanHelpers.h
+3-3llvm/lib/Transforms/Vectorize/VPlan.cpp
+16-142 files

LLVM/project 275baedllvm/lib/Analysis LoopAccessAnalysis.cpp, llvm/test/Analysis/LoopAccessAnalysis underlying-object-different-address-spaces.ll

[LAA] Consider accessed addrspace when mapping underlying obj to access. (#129087)

In some cases, it is possible for the same underlying object to be
accessed via pointers to different address spaces. This could lead to
pointers from different address spaces ending up in the same dependency
set, which isn't allowed (and triggers an assertion).

Update the mapping from underlying object -> last access to also include
the accessing address space.

Fixes https://github.com/llvm/llvm-project/issues/124759.

PR: https://github.com/llvm/llvm-project/pull/129087
DeltaFile
+39-0llvm/test/Analysis/LoopAccessAnalysis/underlying-object-different-address-spaces.ll
+8-4llvm/lib/Analysis/LoopAccessAnalysis.cpp
+47-42 files

LLVM/project f9b2497llvm/lib/Transforms/Vectorize VPlanHelpers.h

[VPlan] Use const for VPBasicBlock* in key in VPBB2IRBB (NFC).

This allows queries in places where only a const pointer to VPBasiBlocks
is available.
DeltaFile
+1-1llvm/lib/Transforms/Vectorize/VPlanHelpers.h
+1-11 files

LLVM/project c0bf4b2llvm/lib/Transforms/Vectorize VPlanValue.h

[VPlan] Remove unneeded VPValue::getLiveInIRValue() const (NFC).

The accessor is not needed/used.
DeltaFile
+1-6llvm/lib/Transforms/Vectorize/VPlanValue.h
+1-61 files

LLVM/project 6ce41dbllvm/lib/Transforms/Vectorize VPlanRecipes.cpp VPlan.h, llvm/test/Transforms/LoopVectorize debugloc.ll

[VPlan] Preserve DebugLoc for VPBranchOnMaskRecipe.

Update code to set and generate debug location for branch recipe
DeltaFile
+4-2llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
+3-3llvm/lib/Transforms/Vectorize/VPlan.h
+2-4llvm/test/Transforms/LoopVectorize/debugloc.ll
+3-1llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
+2-2llvm/unittests/Transforms/Vectorize/VPlanTest.cpp
+14-125 files

LLVM/project 253d691llvm/lib/Transforms/Vectorize VPlan.h VPlanRecipes.cpp

[VPlan] Update VPBranchOnMaskRecipe to always set the mask (NFC).

The mask is always available at construction time. Make it non-optional
to simlpify code.
DeltaFile
+2-16llvm/lib/Transforms/Vectorize/VPlan.h
+2-7llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
+4-232 files

LLVM/project 1e1b9bcllvm/lib/Transforms/Vectorize VPlanTransforms.cpp, llvm/test/Transforms/LoopVectorize if-pred-non-void.ll induction.ll

[VPlan] Simplify BLEND %a, %b, NOT(%m) -> BLEND %b, %a, %m. (#128375)

Avoid negations for normalized blends by reordering operands.

PR: https://github.com/llvm/llvm-project/pull/128375
DeltaFile
+15-21llvm/test/Transforms/LoopVectorize/if-pred-non-void.ll
+11-16llvm/test/Transforms/LoopVectorize/induction.ll
+11-12llvm/test/Transforms/LoopVectorize/load-of-struct-deref-pred.ll
+7-10llvm/test/Transforms/LoopVectorize/if-pred-stores.ll
+14-0llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
+4-7llvm/test/Transforms/LoopVectorize/X86/drop-poison-generating-flags.ll
+62-6613 files not shown
+88-10919 files

LLVM/project 3afc3f4llvm/lib/Transforms/IPO MergeFunctions.cpp, llvm/test/Transforms/MergeFunc merge-linkonce-odr.ll linkonce.ll

[MergeFunc] Remove discardables function before writing alias or thunk. (#128865)

Update writeThunkOrAlias to only create an alias or thunk if it is
actually needed. Drop discardable linkone_odr functions if they are not
used before.

PR: https://github.com/llvm/llvm-project/pull/128865
DeltaFile
+21-14llvm/lib/Transforms/IPO/MergeFunctions.cpp
+1-13llvm/test/Transforms/MergeFunc/merge-linkonce-odr.ll
+2-9llvm/test/Transforms/MergeFunc/linkonce.ll
+2-9llvm/test/Transforms/MergeFunc/linkonce_odr.ll
+26-454 files