Remove items from an array based on another array

Question

Post reply

Remove items from an array based on another array

schleep

SSChampion

Points: 12585
More actions
October 11, 2012 at 7:59 am

#269175

Hi,
I have 2 Array or ArrayList objects, with data like
$Results = New-Object system.Collections.ArrayList
$Results.Add("IMP_Trn_Project_Agent_6,User1")
$Results.Add("IMP_Trn_Project_Agent_6,User2")
$Results.Add("IMP_Trn_Project_Agent_6,User3")
$Remove = New-Object system.Collections.ArrayList
$Remove.Add("User2")
How can I efficiently remove from the first array all items containing the users in the 2nd array?
Paul

Viewing 8 posts - 1 through 7 (of 7 total)

You must be logged in to reply to this topic. Login to reply

bruce 1565 SSCrazy Points: 2574 More actions · Answer 1

Here's one way, there are probably more.

foreach ($result in $results) {if ($remove -contains $result.split(",")[1]) { $results.Remove($result) }}

schleep SSChampion Points: 12585 More actions · Answer 2

I tried your code, it gives me an error:

An error occurred while enumerating through a collection: Collection was modified

; enumeration operation may not execute..

At ps1:10 char:8

+ foreach <<<< ($result in $results) {if ($remove -contains $result.split(",")[1

]) { $results.Remove($result) }}

+ CategoryInfo : InvalidOperation: (System.Collecti...numeratorSimp

le:ArrayListEnumeratorSimple) [], RuntimeException

+ FullyQualifiedErrorId : BadEnumeration

Also, I modified the $Results array to add a second instance of User2: it doesn't get removed, while I need it removed as well.

$Results = New-Object system.Collections.ArrayList

$Results.Add("IMP_Trn_Project_Agent_6,User1")

$Results.Add("IMP_Trn_Project_Agent_6,User2")

$Results.Add("IMP_Trn_Project2_Agent_6,User2")

$Results.Add("IMP_Trn_Project_Agent_6,User3")

$Remove = New-Object system.Collections.ArrayList

$Remove.Add("User2")

foreach ($result in $results) {if ($remove -contains $result.split(",")[1]) { $results.Remove($result) }}

$Results

bruce 1565 SSCrazy Points: 2574 More actions · Answer 3

I must have changed the ErrorActionPreference in the console in which I tested my answer. I didn’t get the error last week but do today.

Anyway, give this version a try. I don’t get errors running with ErrorActionPreference set to Continue.

$Results = New-Object system.Collections.ArrayList

$Results.Add("IMP_Trn_Project_Agent_6,User1")

$Results.Add("IMP_Trn_Project_Agent_6,User2")

$Results.Add("IMP_Trn_Project2_Agent_6,User2")

$Results.Add("IMP_Trn_Project_Agent_6,User3")

$Remove = New-Object system.Collections.ArrayList

$Remove.Add("User2")

for ($i=($Results.Count -1); $i -ge 0; $i--) {

if ($results[$i].split(",")[1] -eq $remove) { $results.RemoveRange($i,1) }

}

$Results

schleep SSChampion Points: 12585 More actions · Answer 4

Thanks again Bruce.

I got it working using a foreach $Result (If not in $Remove, add to $FinalResult) approach.

Works reasonably fast. (13K $Results to check against a list of 500 $Removes)

Cheers!

Paul

bruce 1565 SSCrazy Points: 2574 More actions · Answer 5

OK, that was one of those "other ways" I initially mentioned but I was trying to stay with your original request. Either way I'm glad you got it working.

However, if you find some extra time could you first verify my last reply actually does what you wanted and also compare the time your version and mine take? Or if it's easier to send files with 13K results and 500 removers I'd be glad to do the comparison myself.

Not like it's that important since you've answered your question. I'm just curious about which is more efficient in regard to speed and resources. I'm guessing creating the $FinalResult array is quicker.

schleep SSChampion Points: 12585 More actions · Answer 6

Your way did work with a single entry in $Remove. When I added a second entry, neither was removed??

Weird.

I was also curious as to why you used a for ... i-- ...next, instead of for ... i++ ... next approach?

I didn't investigate further since I already had a working solution.

I wondered if Powershell could "remember" which values it had already looked up in the $Remove list, so that the next time it encountered one already known to be removed, it wouldn't scan the list again. (I think I read somewhere that it can use the equivalent of indexes on sorted data -- don't quote me on that).

It's taking ~4-5 secs to process the 13K records.

So, if I add the desired records to [Array]$FinalList, it takes 15 seconds to pass the array to out-file and write it to disk.

If I instead create a [string]$FinalList, assembling the string takes longer (10-11 seconds), but the out-file is 2 seconds.

And before you ask ;-), the hybrid clean into array, transform to string, and write to file takes the same total amount of time.

What I'm doing is going through AD, getting the memberships for selected groups, and then removing those users which are member of a Disabled OU. I've got this down to 26 seconds, eclipsing my best time of 10+mins using the QAD cmdlets. Since this rarely runs even once a day, I'm OK with that performance.

bruce 1565 SSCrazy Points: 2574 More actions · Answer 7

26 seconds versus 10+ minutes sounds like a winner to me.

The $i—was to go backward through the $Results list since removing an item changed the indexing causing the errors and missing the duplicate “User2” entries.

I don’t think I helped you at all but appreciate your feedback.