Thread: [Asteria 3.0] Npc Drop Dumper [RS WIKIA]

Results 1 to 8 of 8
  1. #1 [Asteria 3.0] Npc Drop Dumper [RS WIKIA] 
    Registered Member
    Cadillac's Avatar
    Join Date
    Jul 2014
    Age
    9
    Posts
    336
    Thanks given
    0
    Thanks received
    228
    Rep Power
    951
    First of all this will require the Jsoup dependency.

    Do not release this modified here or on any other community.

    I am aware this is fairly poorly written, but it does serve it's purpose, and I don't see a need to complicate it as you will probably run it twice at most. If you find any bugs or issues please share. I'm releasing because other drop systems have exploits in them, this is an open source solution.

    Benefits to using this:
    • Very easy to manage and find drops as they are by name
    • No exploits
    • Generates drops faster than other systems
    • Has some configs to start with
    • No data redundancy (not having to write drops for all ids-or any ids for that matter)


    Implementation:

    -Add the dumper to your util package!

    You can skip this step if you just download: Download npc.drops.json @ UppIT

    Spoiler for NpcDropGrabber:
    Code:
    package com.asteria.utility;
    
    import java.io.FileWriter;
    import java.io.IOException;
    import java.util.ArrayList;
    import java.util.HashMap;
    import java.util.LinkedList;
    import java.util.List;
    import java.util.Map;
    import java.util.Optional;
    import java.util.logging.Logger;
    
    import org.jsoup.Jsoup;
    import org.jsoup.nodes.Document;
    import org.jsoup.nodes.Element;
    import org.jsoup.select.Elements;
    
    import com.asteria.game.character.npc.NpcDefinition;
    import com.asteria.game.character.npc.drop.NpcDrop;
    import com.asteria.game.character.npc.drop.NpcDropChance;
    import com.asteria.game.character.npc.drop.NpcDropTable;
    import com.asteria.game.item.ItemDefinition;
    import com.google.gson.Gson;
    import com.google.gson.GsonBuilder;
    
    /**
     * Npc drop grabber - grabs drops from RS WIKI
     * 
     * @author Cadillac 
     */
    public class NpcDropGrabber {
    
    	/* Whether or not to print messages for debugging. */
    	private static boolean debug = true;
    
    	/* The maximum item id */
    	private static final int MAX_ITEM_ID = 20694;
    
    	/* The npc names to grab drops for */
    	private static final List<String> NPC_NAMES = new LinkedList<>();
    
    	/* The logger */
    	Logger log = LoggerUtils.getLogger(NpcDropGrabber.class);
    
    	public void determineNpcs() {
    		for (NpcDefinition def : NpcDefinition.DEFINITIONS) {
    			if (def == null || def.getName() == null) {
    				continue;
    			}
    			if (def.isAttackable() && def.getHitpoints() > 0) {
    				NPC_NAMES.add(def.getName());
    			}
    		}
    	}
    
    	public void run() {
    		determineNpcs();
    
    		// hashmap because I was having touble parsing without specifying
    		HashMap<String, NpcDropTable> drops = new HashMap<>();
    
    		for (String name : NPC_NAMES) {
    			NpcDropTable table = grab(name);
    			if (table == null) {
    				continue;
    			}
    			drops.put(name, table);
    		}
    
    		dump(drops);
    	}
    
    	/**
    	 * Grabs the {@link NpcDropTable} corresponding to the npc name. <br/>
    	 * *** <b> BE SURE TO USE NAMES EXACTLY AS THEY APPEAR IN DEFINITIONS </b>
    	 * 
    	 * @param name
    	 */
    	public NpcDropTable grab(String name) {
    		String key = name;
    
    		// Format the name to append to the url.
    		name = TextUtils.capitalize(name.toLowerCase().replaceAll(" ", "_"));
    
    		List<NpcDrop> dumped = new ArrayList<>();
    		try {
    
    			Document page = Jsoup.connect("http://runescape.wikia.com/wiki/" + name).get();
    
    			System.out.println(page.title());
    
    			// Get the item drop elements (tables) from 'page'
    			Elements dropRows = page.select(".item-drops");
    			//log.info(dropRows.size());
    
    			if (dropRows.size() < 1) {
    				log.info("No drop table for " + name);
    				return null;
    			}
    
    			// Begin looping all of the tables.
    			for (Element row : dropRows) {
    
    				//debug(row);
    
    				// The table itself.
    				Element table = row.children().get(0);
    				//log.info("Amount of children: " + table.children().size());
    
    				// We are at the table rows now and can begin dissecting drops.
    				for (Element tableRow : table.select("tr")) {
    					//debug(tableRow);
    
    					// Size of TD (Table Data) which are the row components http://i.imgur.com/RLc093a.png
    					Elements tableData = tableRow.select("td");
    					//log.info("Size of TD: " + tableData.size());
    
    					// Html which we use for the rarity and amount(s)
    					String html = tableData.html();
    					//log.info("" + html);
    
    					// The a(nchor) element has first, the image, second, the item name
    					Elements links = tableData.select("a");
    					//log.info("Links: " + links.size());
    
    					String itemName = "";
    
    					// for some reason links.get(1) throws an error...
    					int accumulator = 0;
    					for (Element link : links) {
    						if (accumulator == 1) {
    							itemName = link.text();
    						}
    						accumulator++;
    					}
    
    					// Skip if null
    					if (itemName.equals(null)) {
    						continue;
    					}
    
    					Optional<ItemDefinition> def = ItemDefinition.get(itemName);
    
    					// Skip if we don't have a definiton for this item.
    					if (!def.isPresent() || def.get().getId() > MAX_ITEM_ID) {
    						log.info("skipped " + itemName);
    						continue;
    					}
    
    					String[] lines = html.split("\n");
    
    					// The amount of lines where we extract the amoutn and rarity from
    					//log.info("Html Lines " + lines.length);
    
    					String amount = "";
    					String rarity = "";
    
    					int index = 0;
    
    					// Looping because yet again calling lines[5] lines[6] is throwing errors..
    					for (String line : lines) {
    
    						if (index == 5) { // The line id where we get the amount(s)
    							amount = line.replaceAll(",", "");
    						} else if (index == 6) { // The line with rarity
    							rarity = line;
    						}
    
    						index++;
    					}
    
    					// The determined drop chance
    					NpcDropChance chance = NpcDropChance.get(rarity);
    
    					// The item id and note id.
    					int id = def.get().getId();
    					int child = def.get().getNoteId();
    
    					// Now we actually gather the possible drops.
    					if (amount.contains(";")) {
    						String[] split = amount.split(";");
    						for (String s : split) {
    							String trimmed = s.replaceAll(" ", "");
    							// The drop has a varying amount
    							if (s.contains("–")) {
    								String[] values = s.split("–");
    								boolean note = values[0].contains("note") || values[1].contains("note");
    								int min = Integer.parseInt(values[0].replaceAll("[^\\d.]", ""));
    								int max = Integer.parseInt(values[1].replaceAll("[^\\d.]", ""));
    								dumped.add(new NpcDrop(note ? child : id, min, max, chance));
    							// Handle the noted amounts
    							} else if (amount.contains("noted")) {
    								String amt = amount.replaceAll("[^\\d.]", "");
    								int a = Integer.parseInt(amt.replaceAll(" ", ""));
    								dumped.add(new NpcDrop(child, a, a, chance));
    							// An unkown amount so we just set it to 1
    							} else if (amount.equalsIgnoreCase("Unknown")){ 
    								dumped.add(new NpcDrop(id, 1, 1, chance));
    								// The drop has multiple set amounts
    							} else {
    								String amt = s.replaceAll("[^\\d.]", "");
    								if (amt.equals("")) {
    									continue;
    								}
    								int a = Integer.parseInt(amt);
    								dumped.add(new NpcDrop(id, a, a, chance));
    							}
    						}
    						// The dropped item is noted so we drop the noted id
    					} else if (amount.contains("noted")) {
    						String amt = amount.replaceAll("[^\\d.]", "");
    						int a = Integer.parseInt(amt.replaceAll(" ", ""));
    						dumped.add(new NpcDrop(child, a, a, chance));
    						// A single drop with inclusive values
    					} else if (amount.contains("–")) {
    						String[] values = amount.split("–");
    						boolean note = values[0].contains("note") || values[1].contains("note");
    						int min = Integer.parseInt(values[0].replaceAll("[^\\d.]", ""));
    						int max = Integer.parseInt(values[1].replaceAll("[^\\d.]", ""));
    						dumped.add(new NpcDrop(note ? child : id, min, max, chance));
    						// A static drop
    					} else if (amount.equalsIgnoreCase("Unknown")){ 
    						dumped.add(new NpcDrop(id, 1, 1, chance));
    					} else {
    						if (amount.equals("")) {
    							continue;
    						}
    						int a = Integer.parseInt(amount.replaceAll("[^\\d.]", ""));
    						dumped.add(new NpcDrop(id, a, a, chance));
    					}
    
    					log.info("Item Name: " + itemName + " Amount: " + amount + " Rarity : " + rarity);
    					// NpcDrop drop = ItemDefinitions.get(itemName);
    					//List.add(drop);
    				}
    
    			}
    
    		} catch (IOException e) {
    			//System.exit(1);
    			e.printStackTrace();
    		}
    
    		log.info("Drops dumped: " + dumped.size());
    
    		// no drops so this is null.
    		if (dumped.size() < 1) {
    			return null;
    		}
    
    		List<NpcDrop> constants = new ArrayList<>();
    		List<NpcDrop> drops = new ArrayList<>();
    
    		for (NpcDrop drop : dumped) {
    			if (drop.getChance() == NpcDropChance.ALWAYS) {
    				constants.add(drop);
    			} else {
    				drops.add(drop);
    			}
    		}
    
    		NpcDrop[] constantArray = constants.toArray(new NpcDrop[constants.size()]);
    		NpcDrop[] dropArray = drops.toArray(new NpcDrop[drops.size()]);
    
    		return new NpcDropTable(key, constantArray, dropArray);
    	}
    
    	public void dump(HashMap<String, NpcDropTable> map) {
    		// Create a json pretty print object.
    		Gson gson = new GsonBuilder().setPrettyPrinting().create();  
    
    		// 
    		String json = gson.toJson(map);  
    
    		try {  
    			FileWriter writer = new FileWriter("./npc-drops.json");  
    			writer.write(json);  
    			writer.close();  
    
    		} catch (IOException e) {  
    			e.printStackTrace();  
    		}  
    	}
    
    	private void debug(Element element) {
    		if (debug)
    			log.info(element.nodeName()  + " Child Nodes: " + element.childNodeSize() + " Class Name: " + element.className() + " Node Name: " + element.nodeName());
    	}
    
    }


    -Replace the following classes.

    Spoiler for NpcDrop:
    Code:
    package com.asteria.game.character.npc.drop;
    
    import com.asteria.game.item.Item;
    import com.asteria.utility.RandomGen;
    
    /**
     * The container class that represents one NPC drop within a table.
     * 
     * @author lare96 <http://github.com/lare96>
     */
    public final class NpcDrop {
    
        /**
         * The identification of this NPC drop.
         */
        private final int id;
    
        /**
         * The minimum amount that will be dropped.
         */
        private final int minimum;
    
        /**
         * The maximum amount that will be dropped.
         */
        private final int maximum;
    
        /**
         * The chance of this item being dropped.
         */
        private final NpcDropChance chance;
    
        /**
         * Creates a new {@link NpcDrop}.
         * 
         * @param id
         *            the identification of this NPC drop.
         * @param minimum
         *            the minimum amount that will be dropped.
         * @param maximum
         *            the maximum amount that will be dropped.
         * @param chance
         *            the chance of this item being dropped.
         */
        public NpcDrop(int id, int minimum, int maximum, NpcDropChance chance) {
            this.id = id;
            this.minimum = minimum;
            this.maximum = maximum;
            this.chance = chance;
        }
    
        /**
         * The random generator instance that will generate random numbers.
         */
        private static RandomGen random = new RandomGen();
        
        /**
         * Converts this NPC drop into an {@link Item} object.
         * 
         * @param random
         *            the random number generator to use.
         * @return the converted NPC drop.
         */
        public Item toItem() {
            return new Item(id, random.inclusive(minimum, maximum));
        }
    
        /**
         * Gets the identification of this NPC drop.
         * 
         * @return the identification.
         */
        public int getId() {
            return id;
        }
    
        /**
         * Gets the minimum amount that will be dropped.
         * 
         * @return the minimum amount.
         */
        public int getMinimum() {
            return minimum;
        }
    
        /**
         * Gets the maximum amount that will be dropped.
         * 
         * @return the maximum amount.
         */
        public int getMaximum() {
            return maximum;
        }
    
        /**
         * Gets the chance of this item being dropped.
         * 
         * @return the drop chance.
         */
        public NpcDropChance getChance() {
            return chance;
        }
    }


    Spoiler for NpcDropTable:
    Code:
    package com.asteria.game.character.npc.drop;
    
    import java.io.BufferedReader;
    import java.io.FileReader;
    import java.util.ArrayList;
    import java.util.Collections;
    import java.util.HashMap;
    import java.util.List;
    import java.util.Map;
    
    import com.asteria.game.character.player.Player;
    import com.asteria.game.item.Item;
    import com.asteria.utility.DefinitionBuilder;
    import com.asteria.utility.RandomGen;
    import com.google.gson.Gson;
    import com.google.gson.reflect.TypeToken;
    
    /**
     * The container class that contains the drop tables for a set of NPCs along
     * with functions to manage these tables.
     * 
     * @author lare96 <http://github.com/lare96>
     * @author Cadillac <http://bitbucket.org/cadillac-rs>
     */
    public final class NpcDropTable {
    
        /**
         * The hash collection of drop tables for all NPCs.
         */
        public static final Map<String, NpcDropTable> DROPS = new HashMap<>();
        
        /**
         * The name of npcs who will drop these items.
         */
        private final String name;
        
        /**
         * The constant {@link Item}s which will <b>ALWAYS</b> be dropped.
         */
        private final NpcDrop[] constants;
        
        /**
         * The <b>chance</b> items which will be recieved according to {@link NpcDropChance} coordinated rates.
         */
        private final NpcDrop[] drops;
    
    	public NpcDropTable(String name, NpcDrop[] constants, NpcDrop[] drops) {
    		this.name = name;
    		this.constants = constants;
    		this.drops = drops;
    	}
    
    	/**
         * Performs the necessary calculations on all of the tables in this
         * container to determine an array of items to drop. Please note that this
         * is not a static implementation meaning that calling this multiple times
         * will return a different list of items.
         * 
         * @param player
         *            the player that these calculations are being performed for.
         * @return the list of items that were calculated.
         */
        public NpcDrop[] toItems(Player player) {
        	// A list to temporarily contain all drops.
            List<NpcDrop> items = new ArrayList<>();
            Collections.addAll(items, constants);
            
            // Collect successful drops
            for (NpcDrop drop : drops)  {
                if (drop.getChance().successful()) {
                    items.add(drop);
                }
            }
            
            // Return the array
            return items.toArray(new NpcDrop[items.size()]);
        }
    	
    	public String getName() {
    		return name;
    	}
    
    	public NpcDrop[] getConstants() {
    		return constants;
    	}
    
    	public NpcDrop[] getDrops() {
    		return drops;
    	}
        
    	/**
    	 * A utility for building all the drops at startup.
    	 *
    	 * @author Cadillac <http://bitbucket.org/cadillac-rs>
    	 */
    	public static final class NpcDropBuilder extends DefinitionBuilder {
    
    		/**
    		 * The method we use to get and read the json file which has all npc drops.
    		 * 
    		 * @return
    		 */
    		private HashMap<String, NpcDropTable> deserialize() {
    			Gson gson = new Gson();
    			try (BufferedReader reader = new BufferedReader(new FileReader("./data/json/npcs/npc-drops.json"));) {
    				return gson.fromJson(reader, new TypeToken<Map<String, NpcDropTable>>(){}.getType());
    			} catch ( Exception e) {
    				logger().info("Unable to build Npc Drops");
    				System.exit(0);
    			}
    			return null;
    		}
    
    	    @Override
    	    public void build() {
    	       DROPS.putAll(deserialize());
    	    }
    	}
    }


    Spoiler for NpcDropChance:
    Code:
    package com.asteria.game.character.npc.drop;
    
    import com.asteria.utility.RandomGen;
    
    /**
     * The enumerated type whose elements represent the NPC drop rates.
     * 
     * @author lare96 <http://github.com/lare96>
     */
    public enum NpcDropChance {
        ALWAYS(1, 1) {
            @Override
            public boolean successful() {
                return true;
            }
        },
        COMMON(2, 50),
        UNCOMMON(51, 100),
        RARE(101, 512),
        VERY_RARE(513, 1000);
    
        /**
         * The numerator of this NPC drop rate.
         */
        private final int numerator;
    
        /**
         * The denominator of this NPC drop rate.
         */
        private final int denominator;
    
        /**
         * Creates a new {@link NpcDropChance}.
         * 
         * @param numerator
         *            the numerator of this NPC drop rate.
         * @param denominator
         *            the denominator of this NPC drop rate.
         */
        private NpcDropChance(int numerator, int denominator) {
            this.numerator = numerator;
            this.denominator = denominator;
        }
        
        /**
         * The random generator instance that will generate random numbers.
         */
        private static RandomGen random = new RandomGen();
    
        /**
         * Determines if an NPC drop will be successful or not.
         * 
         * @param random
         *            the random number generator used to determine this.
         * @return {@code true} if the drop was successful, {@code false} otherwise.
         */
        public boolean successful() {
            return (random.inclusive(numerator, denominator)) % numerator == 0;
        }
    
        /**
         * Gets the numerator of this NPC drop rate.
         * 
         * @return the numerator.
         */
        public final int getNumerator() {
            return numerator;
        }
    
        /**
         * Gets the denominator of this NPC drop rate.
         * 
         * @return the denominator.
         */
        public final int getDenominator() {
            return denominator;
        }
        
        public static NpcDropChance get(String string) {
        	for (NpcDropChance c : values()) {
        		if (c.name().toLowerCase() == string.toLowerCase().replaceAll(" ", "_")) {
        			return c;
        		}
        	}
        	return RARE;
        }
        
    }


    Replace the portion of death() method with this

    Spoiler for NpcDeath:
    Code:
     Optional<NpcDropTable> drops = Optional.ofNullable(NpcDropTable.DROPS.get(character.getDefinition().getName()));
            drops.ifPresent(t -> {
                NpcDrop[] dropItems = t.toItems(killer.orElse(null));
                for (NpcDrop drop : dropItems) {
                    if (drop == null)
                        continue;
                    ItemNodeManager.register(!killer.isPresent() ? new ItemNodeStatic(drop.toItem(), character.getPosition()) : new ItemNode(
                        drop.toItem(), character.getPosition(), killer.get()));
                }
                killer.ifPresent(k -> MinigameHandler.search(k).ifPresent(m -> m.onKill(k, character)));
            });


    In ServerBuilder add:

    Code:
    new NpcDropGrabber().run();
    above: (put it here because then it will load after item and npc definitions.

    Code:
    sequencer.scheduleAtFixedRate(new GameService(), 0, 600, TimeUnit.MILLISECONDS);
    Now you should have a new file in your server folder npc-drops.json

    put it in data/json/npcs/

    And to load at startup:
    Code:
    import com.asteria.game.character.npc.drop.NpcDropTable.NpcDropBuilder;
    Code:
    serviceLoader.execute(() -> new NpcDropBuilder().build());
    and remove
    Code:
    new NpcDropGrabber().run();
    Should be all, message me if you need method or anything is not working. The only issue I've had is getting a 410 exception due to too much traffic from your machine, this can be mitigated be putting a delay between each grab.

    If you need help please leave a reply.

    Only Lare has permission to rerelease this.
    back at it.
    Reply With Quote  
     

  2. Thankful users:


  3. #2  
    Registered Member
    Stanaveli's Avatar
    Join Date
    Aug 2014
    Posts
    1,490
    Thanks given
    184
    Thanks received
    653
    Rep Power
    1338
    Goodjob mate
    Keep your head up.



    Reply With Quote  
     

  4. #3  
    Registered Member
    Cadillac's Avatar
    Join Date
    Jul 2014
    Age
    9
    Posts
    336
    Thanks given
    0
    Thanks received
    228
    Rep Power
    951
    Quote Originally Posted by Stand up View Post
    Goodjob mate
    Thanks, I feel a lot of people could find this useful!
    back at it.
    Reply With Quote  
     

  5. Thankful user:


  6. #4  
    Banned
    Join Date
    Aug 2014
    Posts
    518
    Thanks given
    88
    Thanks received
    21
    Rep Power
    0
    Good job might use in the future Thanks
    Reply With Quote  
     

  7. #5  
    freelance vagina illustrator
    Plexus's Avatar
    Join Date
    Apr 2014
    Age
    25
    Posts
    288
    Thanks given
    190
    Thanks received
    106
    Rep Power
    407
    repped thanks i love you have my babies
    I Support
    Maiestas
    Reply With Quote  
     

  8. #6  
    oof


    Join Date
    Aug 2012
    Posts
    3,150
    Thanks given
    2,847
    Thanks received
    857
    Rep Power
    2260
    Thanks
    Reply With Quote  
     

  9. #7  
    Registered Member

    Join Date
    Nov 2014
    Posts
    337
    Thanks given
    47
    Thanks received
    12
    Rep Power
    187
    Could you reupload link, its not there now.
    Reply With Quote  
     

  10. #8  
    Owner of Dawntained

    Mgt Madness's Avatar
    Join Date
    Oct 2011
    Age
    28
    Posts
    3,380
    Thanks given
    1,429
    Thanks received
    958
    Rep Power
    2168
    The wiki parser is all i needed, this is amazing work, gj man
    Attached image
    Reply With Quote  
     


Thread Information
Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)


User Tag List

Similar Threads

  1. Replies: 5
    Last Post: 08-18-2014, 05:55 PM
  2. Replies: 1
    Last Post: 11-26-2013, 03:54 PM
  3. 718 Npc Drop DUmper REQ
    By crowley in forum Requests
    Replies: 2
    Last Post: 08-02-2013, 06:20 PM
  4. Replies: 3
    Last Post: 06-22-2012, 01:43 PM
  5. 100% real rs npc drops MUST SEE!!!
    By tokyomewmew in forum Show-off
    Replies: 53
    Last Post: 11-23-2009, 09:21 PM
Posting Permissions
  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •