How do I remove automatically remove repeated words

8
Found 23rd Nov 2017
I have a big list of words. Roughly around 10000 words in a list. Was wondering if there is a method to remove any repeated words. Manually will take forever.

Thanks in advance
Community Updates
Ask
8 Comments
what format is the list in?

If it's excel, just click the "remove duplicates" button.
Install Notepad++

Then go to Plugins → Plugin Manager → Show Plugin Manager, then install TextFX

Once that's done do the following (do it on a backup copy not the original)

Mark all the text (CTRL+A) TextFX → TextFX Tools → Check +Sort outputs only UNIQUE (at column) lines (if not already checked).

Then do TextFX → TextFX Tools → Sort lines case insensitive (at column)

Job Done

(Disable Sort ascending first if you don't want them in alphabetical order )

TextFX → TextFX Tools → +Sort ascending
Edited by: "4Real2016" 23rd Nov 2017
As first comment if it’s a list put it in excel and data remove duplicar
As others are saying you have not given the format of the list.

If using windows then without any other tools powershell can do this with one line example being:

get-content "e:\test.txt" | Select-Object -Unique |out-file "e:\result.txt"

This is assuming of course that each word entry in your list is on its own line, if not powershell would still be able to handle but the import and piping out may need a little bit of work.
Edited by: "Bertz99" 23rd Nov 2017
Bertz9923rd Nov 2017

As others are saying you have not given the format of the list.If using …As others are saying you have not given the format of the list.If using windows then without any other tools powershell can do this with one line example being:get-content "e:\test.txt" | Select-Object -Unique |out-file "e:\result.txt" This is assuming of course that each word entry in your list is on its own line, if not powershell would still be able to handle but the import and piping out may need a little bit of work.


It’s a word document of the list
bullet9t91 h, 5 m ago

It’s a word document of the list



If each word is on a separate line you could save as plain text and then run command I gave above changing to your source file and result file you want.

As you are quoting word good chance you have excel and can cut and paste it to go with the first answer given by Adam.

If you have deliminated another way then you can special import into excel to take account of that and export again to revert - or search powershell examples to expand on my example to cater for your own construction.
Edited by: "Bertz99" 23rd Nov 2017
bullet9t913 h, 28 m ago

It’s a word document of the list


In word, run the grammar check which will highlight repeated words
Post a comment
Avatar
@
    Text

    Top Discussions

    Top Discussions

    Top Merchants